Mailman 3 May 2013 - Python-Dev

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com May 18, 2021

May 18, 2021

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: $Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

8 14

More compact dictionaries with faster iteration
by Raymond Hettinger Jan. 3, 2015

Jan. 3, 2015

The current memory layout for dictionaries is unnecessarily inefficient. It has a sparse table of 24-byte entries containing the hash value, key pointer, and value pointer. Instead, the 24-byte entries should be stored in a dense table referenced by a sparse table of indices. For example, the dictionary: d = {'timmy': 'red', 'barry': 'green', 'guido': 'blue'} is currently stored as: entries = [['--', '--', '--'], [-8522787127447073495, 'barry', 'green'], ['--', '--', '--'], ['--', '--', '--'], ['--', '--', '--'], [-9092791511155847987, 'timmy', 'red'], ['--', '--', '--'], [-6480567542315338377, 'guido', 'blue']] Instead, the data should be organized as follows: indices = [None, 1, None, None, None, 0, None, 2] entries = [[-9092791511155847987, 'timmy', 'red'], [-8522787127447073495, 'barry', 'green'], [-6480567542315338377, 'guido', 'blue']] Only the data layout needs to change. The hash table algorithms would stay the same. All of the current optimizations would be kept, including key-sharing dicts and custom lookup functions for string-only dicts. There is no change to the hash functions, the table search order, or collision statistics. The memory savings are significant (from 30% to 95% compression depending on the how full the table is). Small dicts (size 0, 1, or 2) get the most benefit. For a sparse table of size t with n entries, the sizes are: curr_size = 24 * t new_size = 24 * n + sizeof(index) * t In the above timmy/barry/guido example, the current size is 192 bytes (eight 24-byte entries) and the new size is 80 bytes (three 24-byte entries plus eight 1-byte indices). That gives 58% compression. Note, the sizeof(index) can be as small as a single byte for small dicts, two bytes for bigger dicts and up to sizeof(Py_ssize_t) for huge dict. In addition to space savings, the new memory layout makes iteration faster. Currently, keys(), values, and items() loop over the sparse table, skipping-over free slots in the hash table. Now, keys/values/items can loop directly over the dense table, using fewer memory accesses. Another benefit is that resizing is faster and touches fewer pieces of memory. Currently, every hash/key/value entry is moved or copied during a resize. In the new layout, only the indices are updated. For the most part, the hash/key/value entries never move (except for an occasional swap to fill a hole left by a deletion). With the reduced memory footprint, we can also expect better cache utilization. For those wanting to experiment with the design, there is a pure Python proof-of-concept here: http://code.activestate.com/recipes/578375 YMMV: Keep in mind that the above size statics assume a build with 64-bit Py_ssize_t and 64-bit pointers. The space savings percentages are a bit different on other builds. Also, note that in many applications, the size of the data dominates the size of the container (i.e. the weight of a bucket of water is mostly the water, not the bucket). Raymond

27 70

Reviving restricted mode?
by Guido van Rossum Aug. 14, 2014

Aug. 14, 2014

I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug: http://code.google.com/p/googleappengine/issues/detail?id=671 Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.) I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

19 35

CLA link from bugs.python.org
by Tim Delaney Feb. 16, 2014

Feb. 16, 2014

It appears there's no obvious link from bugs.python.org to the contributor agreement - you need to go via the unintuitive link Foundation -> Contribution Forms (and from what I've read, you're prompted when you add a patch to the tracker). I'd suggest that if the "Contributor Form Received" field is "No" in user details, there be a link to http://www.python.org/psf/contrib/. Tim Delaney

3 4

cffi in stdlib
by Maciej Fijalkowski Dec. 19, 2013

Dec. 19, 2013

Hello. I would like to discuss on the language summit a potential inclusion of cffi[1] into stdlib. This is a project Armin Rigo has been working for a while, with some input from other developers. It seems that the main reason why people would prefer ctypes over cffi these days is "because it's included in stdlib", which is not generally the reason I would like to hear. Our calls to not use C extensions and to use an FFI instead has seen very limited success with ctypes and quite a lot more since cffi got released. The API is fairly stable right now with minor changes going in and it'll definitely stablize until Python 3.4 release. Notable projects using it: * pypycore - gevent main loop ported to cffi * pgsql2cffi * sdl-cffi bindings * tls-cffi bindings * lxml-cffi port * pyzmq * cairo-cffi * a bunch of others So relatively a lot given that the project is not even a year old (it got 0.1 release in June). As per documentation, the advantages over ctypes: * The goal is to call C code from Python. You should be able to do so without learning a 3rd language: every alternative requires you to learn their own language (Cython, SWIG) or API (ctypes). So we tried to assume that you know Python and C and minimize the extra bits of API that you need to learn. * Keep all the Python-related logic in Python so that you don’t need to write much C code (unlike CPython native C extensions). * Work either at the level of the ABI (Application Binary Interface) or the API (Application Programming Interface). Usually, C libraries have a specified C API but often not an ABI (e.g. they may document a “struct” as having at least these fields, but maybe more). (ctypes works at the ABI level, whereas Cython and native C extensions work at the API level.) * We try to be complete. For now some C99 constructs are not supported, but all C89 should be, including macros (and including macro “abuses”, which you can manually wrap in saner-looking C functions). * We attempt to support both PyPy and CPython, with a reasonable path for other Python implementations like IronPython and Jython. * Note that this project is not about embedding executable C code in Python, unlike Weave. This is about calling existing C libraries from Python. so among other things, making a cffi extension gives you the same level of security as writing C (and unlike ctypes) and brings quite a bit more flexibility (API vs ABI issue) that let's you wrap arbitrary libraries, even those full of macros. Cheers, fijal .. [1]: http://cffi.readthedocs.org/en/release-0.5/

22 75

compile python 3.3 with bz2 support
by Isml Oct. 22, 2013

Oct. 22, 2013

hi, everyone: I want to compile python 3.3 with bz2 support on RedHat 5.5 but fail to do that. Here is how I do it: 1、download bzip2 and compile it(make、make -f Makefile_libbz2_so、make install) 2、chang to python 3.3 source directory : ./configure --with-bz2=/usr/local/include 3、make 4、make install after installation complete, I test it： [root@localhost Python-3.3.0]# python3 -c "import bz2" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python3.3/bz2.py", line 21, in <module> from _bz2 import BZ2Compressor, BZ2Decompressor ImportError: No module named '_bz2' By the way, RedHat 5.5 has a built-in python 2.4.3. Would it be a problem?

3 2

Make extension module initialisation more like Python module initialisation
by Stefan Behnel Aug. 6, 2013

Aug. 6, 2013

Hi, I suspect that this will be put into a proper PEP at some point, but I'd like to bring this up for discussion first. This came out of issues 13429 and 16392. http://bugs.python.org/issue13429 http://bugs.python.org/issue16392 Stefan The problem =========== Python modules and extension modules are not being set up in the same way. For Python modules, the module is created and set up first, then the module code is being executed. For extensions, i.e. shared libraries, the module init function is executed straight away and does both the creation and initialisation. This means that it knows neither the __file__ it is being loaded from nor its package (i.e. its FQMN). This hinders relative imports and resource loading. In Py3, it's also not being added to sys.modules, which means that a (potentially transitive) re-import of the module will really try to reimport it and thus run into an infinite loop when it executes the module init function again. And without the FQMN, it's not trivial to correctly add the module to sys.modules either. We specifically run into this for Cython generated modules, for which it's not uncommon that the module init code has the same level of complexity as that of any 'regular' Python module. Also, the lack of a FQMN and correct file path hinders the compilation of __init__.py modules, i.e. packages, especially when relative imports are being used at module init time. The proposal ============ I propose to split the extension module initialisation into two steps in Python 3.4, in a backwards compatible way. Step 1: The current module init function can be reduced to just creating the module instance and returning it (and potentially doing some simple C level setup). Optionally, after creating the module (and this is the new part), the module init code can register a C callback function that will be called after setting up the module. Step 2: The shared library importer receives the module instance from the module init function, adds __file__, __path__, __package__ and friends to the module dict, and then checks for the callback. If non-NULL, it calls it to continue the module initialisation by user code. The callback ============ The callback is defined as follows:: int (*PyModule_init_callback)(PyObject* the_module, PyModuleInitContext* context) "PyModuleInitContext" is a struct that is meant mostly for making the callback more future proof by allowing additional parameters to be passed in. For now, I can see a use case for the following fields:: struct PyModuleInitContext { char* module_name; char* qualified_module_name; } Both names are encoded in UTF-8. As for the file path, I consider it best to retrieve it from the module's __file__ attribute as a Python string object to reduce filename encoding problems. Note that this struct argument is not strictly required, but given that this proposal would have been much simpler if the module init function had accepted such an argument in the first place, I consider it a good idea not to let this chance pass by again. The registration of the callback uses a new C-API function: int PyModule_SetInitFunction(PyObject* module, PyModule_init_callback callback) The function name uses "Set" instead of "Register" to make it clear that there is only one such function per module. An alternative would be a new module creation function "PyModule_Create3()" that takes the callback as third argument, in addition to what "PyModule_Create2()" accepts. This would require users to explicitly pass in the (second) version argument, which might be considered only a minor issue. Implementation ============== The implementation requires local changes to the extension module importer and a new C-API function. In order to store the callback, it should use a new field in the module object struct. Open questions ============== It is not clear how extensions should be handled that register more than one module in their module init function, e.g. compiled packages. One possibility would be to leave the setup to the user, who would have to know all FQMNs anyway in this case, although not the import file path. Alternatively, the import machinery could use a stack to remember for which modules a callback was registered during the last init function call, set up all of them and then call their callbacks. It's not clear if this meets the intention of the user. Alternatives ============ 1) It would be possible to make extension modules optionally export another symbol, e.g. "PyInit2_modulename", that the shared library loader would call in addition to the required function "PyInit_modulename". This would remove the need for a new API that registers the above callback. The drawback is that it also makes it easier to write broken code because a Python version or implementation that does not support this second symbol would simply not call it, without error. The new C-API function would let the build fail instead if it is not supported. 2) The callback could be made available as a Python function in the module dict, thus also removing the need for an explicit registration API. However, this approach would add overhead to both sides, the importer code and the user provided module init code, as it would require additional dictionary handling and the implementation of a one-time Python function in user code. It would also suffer from the problem that missing support in the runtime would pass silently. 3) The callback could be registered statically in the PyModuleDef struct by adding a new field. This is not trivial to do in a backwards compatible way because the struct would grow longer without explicit initialisation by existing user code. Extending PyModuleDef_HEAD_INIT might be possible but would still break at least binary compatibility. 4) Pass a new context argument into the module init function that contains all information necessary to properly and completely set up the module at creation time. This would provide a much simpler and cleaner solution than the proposed solution. However, it will not be possible before Python 4 as it breaks backwards compatibility with all existing extension modules at both the source and binary level.

5 15

PLY in stdlib (was cffi in stdlib)
by David Beazley July 15, 2013

July 15, 2013

> > From: Eli Bendersky <eliben(a)gmail.com> > > I'll be the first one to admit that pycparser is almost certainly not > generally useful enough to be exposed in the stdlib. So just using it as an > implementation detail is absolutely fine. PLY is a more interesting > question, however, since PLY is somewhat more generally useful. That said, > I see all this as implementation details that shouldn't distract us from > the main point of whether cffi should be added. > Regarding the inclusion of PLY or some subcomponent of it in the standard library, it's not an entirely crazy idea in my opinion. LALR(1) parsers have been around for a long time, are generally known to anyone who's used yacc/bison, and would be useful outside the context of cffi or pycparser. PLY has also been around for about 12 years and is what I would call stable. It gets an update about every year or two, but that's about it. PLY is also relatively small--just two files and about 4300 lines of code (much of which could probably be scaled down a bit). The only downside to including PLY might be the fact that there are very few people walking around who've actually had to *implement* an LALR(1) parser generator. Some of the code for that is extremely hairy and mathematical. At this time, I don't think there are any bugs in it, but it's not the sort of thing that one wants to wander into casually. Also, there are some horrible hacks in PLY that I'd really like to get rid of, but am currently stuck with due to backwards compatibility issues. Alex Gaynor has been working on a PLY variant (RPLY) geared at RPython and which has a slightly different programming interface. I'd say if we were to go down this route, he and I should work together to put together some kind of more general "parsing.lalr" package (or similar) that cleans it up and makes it more suitable as a library for building different kinds of parsing tools on top of. Cheers, Dave

9 13

Rough idea for adding introspection information for builtins
by Larry Hastings July 8, 2013

July 8, 2013

The original impetus for Argument Clinic was adding introspection information for builtins--it seemed like any manual approach I came up with would push the builtins maintenance burden beyond the pale. Assuming that we have Argument Clinic or something like it, we don't need to optimize for ease of use from the API end--we can optimize for data size. So the approach writ large: store a blob of data associated with each entry point, as small as possible. Reconstitute the appropriate inspect.Signature on demand by reading that blob. Where to store the data? PyMethodDef is the obvious spot, but I think that structure is part of the stable ABI. So we'd need a new PyMethodDefEx and that'd be a little tiresome. Less violent to the ABI would be defining a new array of pointers-to-introspection-blobs, parallel to the PyMethodDef array, passed in via a new entry point. On to the representation. Consider the function def foo(arg, b=3, *, kwonly='a'): pass I considered four approaches, each listed below along with its total size if it was stored as C static data. 1. A specialized bytecode format, something like pickle, like this: bytes([ PARAMETER_START_LENGTH_3, 'a', 'r', 'g', PARAMETER_START_LENGTH_1, 'b', PARAMETER_DEFAULT_LENGTH_1, '3', KEYWORD_ONLY, PARAMETER_START_LENGTH_6, 'k', 'w', 'o', 'n', 'l', 'y', PARAMETER_DEFAULT_LENGTH_3, '\'', 'a', '\'', END ]) Length: 20 bytes. 2. Just use pickle--pickle the result of inspect.signature() run on a mocked-up signature, just store that. Length: 130 bytes. (Assume a two-byte size stored next to it.) 3. Store a string that, if eval'd, would produce the inspect.Signature. Length: 231 bytes. (This could be made smaller if we could assume "from inspect import *" or "p = inspect.Parameter" or something, but it'd still be easily the heaviest.) 4. Store a string that looks like the Python declaration of the signature, and parse it (Nick's suggestion). For foo above, this would be "(arg,b=3,*,kwonly='a')". Length: 23 bytes. Of those, Nick's suggestion seems best. It's slightly bigger than the specialized bytecode format, but it's human-readable (and human-writable!), and it'd be the easiest to implement. My first idea for implementation: add a "def x" to the front and ": pass" to the end, then run it through ast.parse. Iterate over the tree, converting parameters into inspect.Parameters and handling the return annotation if present. Default values and annotations would be turned into values by ast.eval_literal. (It wouldn't surprise me if there's a cleaner way to do it than the fake function definition; I'm not familiar with the ast module.) We'd want one more mild hack: the DSL will support positional parameters, and inspect.Signature supports positional parameters, so it'd be nice to render that information. But we can't represent that in Python syntax (or at least not yet!), so we can't let ast.parse see it. My suggestion: run it through ast.parse, and if it throws a SyntaxError see if the problem was a slash. If it was, remove the slash, reprocess through ast.parse, and remember that all parameters are positional-only (and barf if there are kwonly, args, or kwargs). Thoughts? //arry/

9 23

PEP 423 : naming conventions and recipes related to packaging
by Benoît Bryon July 5, 2013

July 5, 2013

Hi, Here is an informational PEP proposal: http://hg.python.org/peps/file/52767ab7e140/pep-0423.txt Could you review it for style, consistency and content? Additional notes: * Original discussion posted to distutils-sig(a)python.org * started on May 2012 at http://mail.python.org/pipermail/distutils-sig/2012-May/018551.html * continues in June 2012 at http://mail.python.org/pipermail/distutils-sig/2012-June/018641.html * that's why I set "Discussion-To:<distutils-sig(a)python.org>" header. * Original document was edited as a contrib to cpython documentation: * http://bugs.python.org/issue14899 * file history at https://bitbucket.org/benoitbryon/cpython/history/Doc/packaging/packagename… * but it looked like a PEP, so posted to peps(a)python.org... Regards, Benoit (benoitbb on irc.freenode.net)

11 20

PEP 409 and the stdlib
by Ethan Furman June 21, 2013

June 21, 2013

As a quick reminder, PEP 409 allows this: try: ... except AnError: raise SomeOtherError from None so that if the exception is not caught, we get the traditional single exception traceback, instead of the new: During handling of the above exception, another exception occurred My question: How do we go about putting this in the stdlib? Is this one of the occasions where we don't do it unless we're modifying a module already for some other reason? For that matter, should we? Pros: Makes tracebacks much less confusing, especially coming from a library Cons: Could hide bugs unrelated to what is being caught and transformed -- ~Ethan~

12 32

Ordering keyword dicts
by Guido van Rossum June 10, 2013

June 10, 2013

On Sat, May 18, 2013 at 10:27 PM, Raymond Hettinger <raymond.hettinger(a)gmail.com> wrote: > BTW, I'm +1 on the idea for ordering keyword-args. It makes > it easier to debug if the arguments show-up in the order they > were created. AFAICT, no purpose is served by scrambling them > (which is exacerbated by the new randomized hashing security feature). I'm slow at warming up to the idea. My main concern is speed -- since most code doesn't need it and function calls are already slow (and obviously very common :-) it would be a shame if this slowed down function calls that don't need it noticeably. An observation is that it's only necessary to preserve order if the function definition uses **kwds. AFAIK we currently don't know if this is the case when the call is made though, but perhaps the information could be made available to the call site somehow. There are also many special cases to consider; e.g. using **kwds in the call where kwds is an unordered dict, or calls from C, or calls to C. But maybe someone considers this a challenge and comes up with a patch? The benefits to *some* use cases would be obvious. -- --Guido van Rossum (python.org/~guido)

14 15

PEP 442: Safe object finalization
by Antoine Pitrou June 4, 2013

June 4, 2013

Hello, I would like to submit the following PEP for discussion and evaluation. Regards Antoine. PEP: 442 Title: Safe object finalization Version: $Revision$ Last-Modified: $Date$ Author: Antoine Pitrou <solipsis(a)pitrou.net> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2013-05-18 Python-Version: 3.4 Post-History: Resolution: TBD Abstract ======== This PEP proposes to deal with the current limitations of object finalization. The goal is to be able to define and run finalizers for any object, regardless of their position in the object graph. This PEP doesn't call for any change in Python code. Objects with existing finalizers will benefit automatically. Definitions =========== Reference A directional link from an object to another. The target of the reference is kept alive by the reference, as long as the source is itself alive and the reference isn't cleared. Weak reference A directional link from an object to another, which doesn't keep alive its target. This PEP focusses on non-weak references. Reference cycle A cyclic subgraph of directional links between objects, which keeps those objects from being collected in a pure reference-counting scheme. Cyclic isolate (CI) A reference cycle in which no object is referenced from outside the cycle *and* whose objects are still in a usable, non-broken state: they can access each other from their respective finalizers. Cyclic garbage collector (GC) A device able to detect cyclic isolates and turn them into cyclic trash. Objects in cyclic trash are eventually disposed of by the natural effect of the references being cleared and their reference counts dropping to zero. Cyclic trash (CT) A reference cycle, or former reference cycle, in which no object is referenced from outside the cycle *and* whose objects have started being cleared by the GC. Objects in cyclic trash are potential zombies; if they are accessed by Python code, the symptoms can vary from weird AttributeErrors to crashes. Zombie / broken object An object part of cyclic trash. The term stresses that the object is not safe: its outgoing references may have been cleared, or one of the objects it references may be zombie. Therefore, it should not be accessed by arbitrary code (such as finalizers). Finalizer A function or method called when an object is intended to be disposed of. The finalizer can access the object and release any resource held by the object (for example mutexes or file descriptors). An example is a ``__del__`` method. Resurrection The process by which a finalizer creates a new reference to an object in a CI. This can happen as a quirky but supported side-effect of ``__del__`` methods. Impact ====== While this PEP discusses CPython-specific implementation details, the change in finalization semantics is expected to affect the Python ecosystem as a whole. In particular, this PEP obsoletes the current guideline that "objects with a ``__del__`` method should not be part of a reference cycle". Benefits ======== The primary benefits of this PEP regard objects with finalizers, such as objects with a ``__del__`` method and generators with a ``finally`` block. Those objects can now be reclaimed when they are part of a reference cycle. The PEP also paves the way for further benefits: * The module shutdown procedure may not need to set global variables to None anymore. This could solve a well-known class of irritating issues. The PEP doesn't change the semantics of: * Weak references caught in reference cycles. * C extension types with a custom ``tp_dealloc`` function. Description =========== Reference-counted disposal -------------------------- In normal reference-counted disposal, an object's finalizer is called just before the object is deallocated. If the finalizer resurrects the object, deallocation is aborted. *However*, if the object was already finalized, then the finalizer isn't called. This prevents us from finalizing zombies (see below). Disposal of cyclic isolates --------------------------- Cyclic isolates are first detected by the garbage collector, and then disposed of. The detection phase doesn't change and won't be described here. Disposal of a CI traditionally works in the following order: 1. Weakrefs to CI objects are cleared, and their callbacks called. At this point, the objects are still safe to use. 2. The CI becomes a CT as the GC systematically breaks all known references inside it (using the ``tp_clear`` function). 3. Nothing. All CT objects should have been disposed of in step 2 (as a side-effect of clearing references); this collection is finished. This PEP proposes to turn CI disposal into the following sequence (new steps are in bold): 1. Weakrefs to CI objects are cleared, and their callbacks called. At this point, the objects are still safe to use. 2. **The finalizers of all CI objects are called.** 3. **The CI is traversed again to determine if it is still isolated. If it is determined that at least one object in CI is now reachable from outside the CI, this collection is aborted and the whole CI is resurrected. Otherwise, proceed.** 4. The CI becomes a CT as the GC systematically breaks all known references inside it (using the ``tp_clear`` function). 5. Nothing. All CT objects should have been disposed of in step 4 (as a side-effect of clearing references); this collection is finished. C-level changes =============== Type objects get a new ``tp_finalize`` slot to which ``__del__`` methods are bound. Generators are also modified to use this slot, rather than ``tp_del``. At the C level, a ``tp_finalize`` function is a normal function which will be called with a regular, alive object as its only argument. It should not attempt to revive or collect the object. For compatibility, ``tp_del`` is kept in the type structure. Handling of objects with a non-NULL ``tp_del`` is unchanged: when part of a CI, they are not finalized and end up in ``gc.garbage``. However, a non-NULL ``tp_del`` is not encountered anymore in the CPython source tree (except for testing purposes). On the internal side, a bit is reserved in the GC header for GC-managed objects to signal that they were finalized. This helps avoid finalizing an object twice (and, especially, finalizing a CT object after it was broken by the GC). Discussion ========== Predictability -------------- Following this scheme, an object's finalizer is always called exactly once. The only exception is if an object is resurrected: the finalizer will be called again later. For CI objects, the order in which finalizers are called (step 2 above) is undefined. Safety ------ It is important to explain why the proposed change is safe. There are two aspects to be discussed: * Can a finalizer access zombie objects (including the object being finalized)? * What happens if a finalizer mutates the object graph so as to impact the CI? Let's discuss the first issue. We will divide possible cases in two categories: * If the object being finalized is part of the CI: by construction, no objects in CI are zombies yet, since CI finalizers are called before any reference breaking is done. Therefore, the finalizer cannot access zombie objects, which don't exist. * If the object being finalized is not part of the CI/CT: by definition, objects in the CI/CT don't have any references pointing to them from outside the CI/CT. Therefore, the finalizer cannot reach any zombie object (that is, even if the object being finalized was itself referenced from a zombie object). Now for the second issue. There are three potential cases: * The finalizer clears an existing reference to a CI object. The CI object may be disposed of before the GC tries to break it, which is fine (the GC simply has to be aware of this possibility). * The finalizer creates a new reference to a CI object. This can only happen from a CI object's finalizer (see above why). Therefore, the new reference will be detected by the GC after all CI finalizers are called (step 3 above), and collection will be aborted without any objects being broken. * The finalizer clears or creates a reference to a non-CI object. By construction, this is not a problem. Implementation ============== An implementation is available in branch ``finalize`` of the repository at http://hg.python.org/features/finalize/. Validation ========== Besides running the normal Python test suite, the implementation adds test cases for various finalization possibilities including reference cycles, object resurrection and legacy ``tp_del`` slots. The implementation has also been checked to not produce any regressions on the following test suites: * `Tulip <http://code.google.com/p/tulip/>`_, which makes an extensive use of generators * `Tornado <http://www.tornadoweb.org>`_ * `SQLAlchemy <http://www.sqlalchemy.org/>`_ * `Django <https://www.djangoproject.com/>`_ * `zope.interface <http://pypi.python.org/pypi/zope.interface>`_ References ========== Notes about reference cycle collection and weak reference callbacks: http://hg.python.org/cpython/file/4e687d53b645/Modules/gc_weakref.txt Generator memory leak: http://bugs.python.org/issue17468 Allow objects to decide if they can be collected by GC: http://bugs.python.org/issue9141 Module shutdown procedure based on GC http://bugs.python.org/issue812369 Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

8 21

Problem building Python 2.7.5 with separate sysroot
by Paul Smith June 2, 2013

June 2, 2013

Hi all. I'm trying to build Python 2.7.5 on a GNU/Linux (Linux Mint 14) system, but using a different sysroot (that is, a separate <d>/usr/include, <d>/usr/lib, etc., not the real one for my system). I have shell script wrappers around GCC and its various tools that invoke it with the right paths to force this to happen, and when I call Python's configure I send along "CC=sysroot-gcc", etc. for all the various tools. Note that it's not really a cross-compilation because the target is also a GNU/Linux system on the same hardware architecture. The majority of Python builds just fine like this. However, I'm having serious problems building modules such as fcntl, etc. Looking at the output from the makefile, I can see that somehow, someone is forcibly adding "-I/usr/include/x86_64-linux-gnu" to the link line: building 'termios' extension sysroot-gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/common/sysroot/tools/usr/include -I/common/sysroot/tools/usr/include-fixed -I/usr/local/include -I/home/workspaces/psmith/python/obj/src/Python-2.7.5/Include -I/home/workspaces/psmith/python/obj/bld/python -c /home/workspaces/psmith/python/obj/src/Python-2.7.5/Modules/termios.c -o build/temp.linux-x86_64-2.7/home/workspaces/psmith/python/obj/src/Python-2.7.5/Modules/termios.o This fails miserably because the headers in /usr/include/x86_64-linux-gnu do not play at all nicely with my other sysroot headers. Ditto for other extensions like fcntl, etc. I've searched high and low in the Python source, generated makefiles, config.log, etc. and I cannot find where this -I flag is coming from anywhere. I found the --oldincludedir flag to configure and set it to point into my sysroot as well, but that didn't help: the /usr/include still appears when building these extensions. Can anyone tell me where Python is getting these -I flags and what I need to do to tell it to NOT use those flags when building extensions? I'd also like to remove the -I/usr/local/include, although this is not actually causing me problems right now.

3 4

Re: [Python-Dev] [Python-checkins] cpython: Add a reset_name argument to importlib.util.module_to_load in order to
by Brett Cannon May 31, 2013

May 31, 2013

I realize this broke the buildbots. Missed part of a diff in the commit. I'm trying to split a massive CL into reasonable commit sizes, so please be patient. On Fri, May 31, 2013 at 6:11 PM, brett.cannon <python-checkins(a)python.org>wrote: > http://hg.python.org/cpython/rev/39cc1b04713e > changeset: 83998:39cc1b04713e > user: Brett Cannon <brett(a)python.org> > date: Fri May 31 18:11:17 2013 -0400 > summary: > Add a reset_name argument to importlib.util.module_to_load in order to > control whether to reset the module's __name__ attribute in case a > reload is being done. > > files: > Doc/library/importlib.rst | 6 +++++- > Lib/importlib/_bootstrap.py | 14 +++++++++++++- > Lib/test/test_importlib/test_util.py | 12 ++++++++++++ > 3 files changed, 30 insertions(+), 2 deletions(-) > > > diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst > --- a/Doc/library/importlib.rst > +++ b/Doc/library/importlib.rst > @@ -788,7 +788,7 @@ > > .. versionadded:: 3.3 > > -.. function:: module_to_load(name) > +.. function:: module_to_load(name, *, reset_name=True) > > Returns a :term:`context manager` which provides the module to load. > The > module will either come from :attr:`sys.modules` in the case of > reloading or > @@ -796,6 +796,10 @@ > :attr:`sys.modules` occurs if the module was new and an exception was > raised. > > + If **reset_name** is true and the module requested is being reloaded > then > + the module's :attr:`__name__` attribute will > + be reset to **name**, else it will be left untouched. > + > .. versionadded:: 3.4 > > .. decorator:: module_for_loader > diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py > --- a/Lib/importlib/_bootstrap.py > +++ b/Lib/importlib/_bootstrap.py > @@ -493,8 +493,14 @@ > > """ > > - def __init__(self, name): > + def __init__(self, name, *, reset_name=True): > + """Prepare the context manager. > + > + The reset_name argument specifies whether to unconditionally reset > + the __name__ attribute if the module is found to be a reload. > + """ > self._name = name > + self._reset_name = reset_name > > def __enter__(self): > self._module = sys.modules.get(self._name) > @@ -508,6 +514,12 @@ > # (otherwise an optimization shortcut in import.c becomes > wrong) > self._module.__initializing__ = True > sys.modules[self._name] = self._module > + elif self._reset_name: > + try: > + self._module.__name__ = self._name > + except AttributeError: > + pass > + > return self._module > > def __exit__(self, *args): > diff --git a/Lib/test/test_importlib/test_util.py > b/Lib/test/test_importlib/test_util.py > --- a/Lib/test/test_importlib/test_util.py > +++ b/Lib/test/test_importlib/test_util.py > @@ -55,6 +55,18 @@ > else: > self.fail('importlib.util.module_to_load swallowed an > exception') > > + def test_reset_name(self): > + # If reset_name is true then module.__name__ = name, else leave > it be. > + odd_name = 'not your typical name' > + created_module = imp.new_module(self.module_name) > + created_module.__name__ = odd_name > + sys.modules[self.module_name] = created_module > + with util.module_to_load(self.module_name) as module: > + self.assertEqual(module.__name__, self.module_name) > + created_module.__name__ = odd_name > + with util.module_to_load(self.module_name, reset_name=False) as > module: > + self.assertEqual(module.__name__, odd_name) > + > > class ModuleForLoaderTests(unittest.TestCase): > > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > >

1 0

Summary of Python tracker Issues
by Python tracker May 31, 2013

May 31, 2013

ACTIVITY SUMMARY (2013-05-24 - 2013-05-31) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3997 (+25) closed 25884 (+34) total 29881 (+59) Open issues with patches: 1780 Issues opened (45) ================== #6386: importing yields unexpected results when initial script is a s http://bugs.python.org/issue6386 reopened by ncoghlan #17269: getaddrinfo segfaults on OS X when provided with invalid argum http://bugs.python.org/issue17269 reopened by ned.deily #18050: embedded interpreter or virtualenv fails with "ImportError: ca http://bugs.python.org/issue18050 opened by samueljohn #18052: IDLE 3.3.2 Windows taskbar icon regression http://bugs.python.org/issue18052 opened by terry.reedy #18053: Add checks for Misc/NEWS in make patchcheck http://bugs.python.org/issue18053 opened by Yogesh.Chaudhari #18054: Add more exception related assertions to unittest http://bugs.python.org/issue18054 opened by ncoghlan #18055: Stop using imp in IDLE http://bugs.python.org/issue18055 opened by brett.cannon #18056: Document importlib._bootstrap.NamespaceLoader http://bugs.python.org/issue18056 opened by brett.cannon #18057: Register NamespaceLoader with importlib.abc.Loader http://bugs.python.org/issue18057 opened by brett.cannon #18058: Define is_package for NamespaceLoader http://bugs.python.org/issue18058 opened by brett.cannon #18059: Add multibyte encoding support to pyexpat http://bugs.python.org/issue18059 opened by serhiy.storchaka #18060: Updating _fields_ of a derived struct type yields a bad cif http://bugs.python.org/issue18060 opened by lauri.alanko #18061: m68k Python 3.3 test results http://bugs.python.org/issue18061 opened by mirabilos #18062: m68k FPU precision issue http://bugs.python.org/issue18062 opened by mirabilos #18064: IDLE: add current directory to open_module http://bugs.python.org/issue18064 opened by terry.reedy #18065: set __path__ = [] for frozen packages http://bugs.python.org/issue18065 opened by brett.cannon #18066: Remove SGI-specific code from pty.py http://bugs.python.org/issue18066 opened by akuchling #18068: pickle + weakref.proxy(self) http://bugs.python.org/issue18068 opened by phd #18069: Subprocess picks the wrong executable on Windows http://bugs.python.org/issue18069 opened by berdario #18071: _osx_support compiler_fixup http://bugs.python.org/issue18071 opened by samueljohn #18073: pickle.Unpickler may read too many bytes, causing hangs with b http://bugs.python.org/issue18073 opened by jm #18076: Implement importlib.util.decode_source_bytes() http://bugs.python.org/issue18076 opened by brett.cannon #18078: threading.Condition to allow notify on a specific waiter http://bugs.python.org/issue18078 opened by JBernardo #18081: test_logging failure in WarningsTest on buildbots http://bugs.python.org/issue18081 opened by r.david.murray #18082: Inconsistent behavior of IOBase methods on closed files http://bugs.python.org/issue18082 opened by dwight.guth #18083: _sysconfigdata.py is installed in an arch-independent director http://bugs.python.org/issue18083 opened by automatthias #18085: Verifying refcounts.dat http://bugs.python.org/issue18085 opened by serhiy.storchaka #18088: Create importlib.abc.Loader.init_module_attrs() http://bugs.python.org/issue18088 opened by brett.cannon #18089: Create importlib.abc.InspectLoader.load_module() http://bugs.python.org/issue18089 opened by brett.cannon #18090: dict_contains first argument declared register, and shouldn't http://bugs.python.org/issue18090 opened by larry #18091: Remove PyNoArgsFunction http://bugs.python.org/issue18091 opened by larry #18092: Python 2.7.5 installation broken on OpenSuse 12.2 http://bugs.python.org/issue18092 opened by Andreas.Jung #18093: Move main functions to a separate Programs directory http://bugs.python.org/issue18093 opened by ncoghlan #18094: Skip tests in test_uuid not silently http://bugs.python.org/issue18094 opened by serhiy.storchaka #18095: unable to invoke socket.connect with AF_UNSPEC http://bugs.python.org/issue18095 opened by Roman.Valov #18096: bad library order returned by python-config.in http://bugs.python.org/issue18096 opened by taylor #18099: wsgiref sets Content-Length: 0 on 304 Not Modified http://bugs.python.org/issue18099 opened by flox #18100: socket.sendall() cannot send buffers of 2GB or more http://bugs.python.org/issue18100 opened by Tom.van.Leeuwen #18101: Tk.split() doesn't work with nested Unicode strings http://bugs.python.org/issue18101 opened by serhiy.storchaka #18102: except-clause with own exception class inside generator can le http://bugs.python.org/issue18102 opened by hagen #18103: Create a GUI test framework for Idle http://bugs.python.org/issue18103 opened by terry.reedy #18104: Idle: make human-mediated GUI tests usable http://bugs.python.org/issue18104 opened by terry.reedy #18105: ElementTree writes invalid files when UTF-16 encoding is speci http://bugs.python.org/issue18105 opened by Adam.Urban #18106: There are unused variables in Lib/test/test_collections.py http://bugs.python.org/issue18106 opened by vajrasky #18108: shutil.chown should support dir_fd and follow_symlinks keyword http://bugs.python.org/issue18108 opened by cjwatson Most recent 15 issues with no replies (15) ========================================== #18108: shutil.chown should support dir_fd and follow_symlinks keyword http://bugs.python.org/issue18108 #18101: Tk.split() doesn't work with nested Unicode strings http://bugs.python.org/issue18101 #18099: wsgiref sets Content-Length: 0 on 304 Not Modified http://bugs.python.org/issue18099 #18096: bad library order returned by python-config.in http://bugs.python.org/issue18096 #18095: unable to invoke socket.connect with AF_UNSPEC http://bugs.python.org/issue18095 #18094: Skip tests in test_uuid not silently http://bugs.python.org/issue18094 #18089: Create importlib.abc.InspectLoader.load_module() http://bugs.python.org/issue18089 #18088: Create importlib.abc.Loader.init_module_attrs() http://bugs.python.org/issue18088 #18082: Inconsistent behavior of IOBase methods on closed files http://bugs.python.org/issue18082 #18081: test_logging failure in WarningsTest on buildbots http://bugs.python.org/issue18081 #18076: Implement importlib.util.decode_source_bytes() http://bugs.python.org/issue18076 #18073: pickle.Unpickler may read too many bytes, causing hangs with b http://bugs.python.org/issue18073 #18068: pickle + weakref.proxy(self) http://bugs.python.org/issue18068 #18066: Remove SGI-specific code from pty.py http://bugs.python.org/issue18066 #18065: set __path__ = [] for frozen packages http://bugs.python.org/issue18065 Most recent 15 issues waiting for review (15) ============================================= #18106: There are unused variables in Lib/test/test_collections.py http://bugs.python.org/issue18106 #18101: Tk.split() doesn't work with nested Unicode strings http://bugs.python.org/issue18101 #18094: Skip tests in test_uuid not silently http://bugs.python.org/issue18094 #18093: Move main functions to a separate Programs directory http://bugs.python.org/issue18093 #18078: threading.Condition to allow notify on a specific waiter http://bugs.python.org/issue18078 #18066: Remove SGI-specific code from pty.py http://bugs.python.org/issue18066 #18059: Add multibyte encoding support to pyexpat http://bugs.python.org/issue18059 #18055: Stop using imp in IDLE http://bugs.python.org/issue18055 #18049: Re-enable threading test on OSX http://bugs.python.org/issue18049 #18045: get_python_version is not import in bdist_rpm.py http://bugs.python.org/issue18045 #18038: Unhelpful error message on invalid encoding specification http://bugs.python.org/issue18038 #18033: Example for Profile Module shows incorrect method http://bugs.python.org/issue18033 #18020: html.escape 10x slower than cgi.escape http://bugs.python.org/issue18020 #18015: python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or http://bugs.python.org/issue18015 #18013: cgi.FieldStorage does not parse W3C sample http://bugs.python.org/issue18013 Top 10 most discussed issues (10) ================================= #16832: Expose cache validity checking support in ABCMeta http://bugs.python.org/issue16832 19 msgs #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 16 msgs #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 12 msgs #18062: m68k FPU precision issue http://bugs.python.org/issue18062 12 msgs #18078: threading.Condition to allow notify on a specific waiter http://bugs.python.org/issue18078 9 msgs #1693050: \w not helpful for non-Roman scripts http://bugs.python.org/issue1693050 9 msgs #5124: IDLE - pasting text doesn't delete selection http://bugs.python.org/issue5124 8 msgs #12641: Remove -mno-cygwin from distutils http://bugs.python.org/issue12641 8 msgs #18085: Verifying refcounts.dat http://bugs.python.org/issue18085 8 msgs #18090: dict_contains first argument declared register, and shouldn't http://bugs.python.org/issue18090 8 msgs Issues closed (34) ================== #8240: ssl.SSLSocket.write may fail on non-blocking sockets http://bugs.python.org/issue8240 closed by pitrou #15392: Create a unittest framework for IDLE http://bugs.python.org/issue15392 closed by terry.reedy #17206: Py_XDECREF() expands its argument multiple times http://bugs.python.org/issue17206 closed by python-dev #17272: request.full_url: unexpected results on assignment http://bugs.python.org/issue17272 closed by orsenthil #17700: Update Curses HOWTO for 3.4 http://bugs.python.org/issue17700 closed by akuchling #17746: test_shutil.TestWhich.test_non_matching_mode fails when runnin http://bugs.python.org/issue17746 closed by serhiy.storchaka #17768: _decimal: allow NUL fill character http://bugs.python.org/issue17768 closed by skrah #17953: sys.modules cannot be reassigned http://bugs.python.org/issue17953 closed by brett.cannon #18009: os.write.__doc__ is misleading http://bugs.python.org/issue18009 closed by python-dev #18011: Inconsistency between b32decode() documentation, docstring and http://bugs.python.org/issue18011 closed by serhiy.storchaka #18025: Buffer overflow in BufferedIOBase.readinto() http://bugs.python.org/issue18025 closed by serhiy.storchaka #18040: SIGINT catching regression on windows in 2.7 http://bugs.python.org/issue18040 closed by tim.golden #18046: Simplify and clarify logging internals http://bugs.python.org/issue18046 closed by python-dev #18047: Descriptors get invoked in old-style objects and classes http://bugs.python.org/issue18047 closed by rhettinger #18051: get_python_version undefined in bdist_rpm.py http://bugs.python.org/issue18051 closed by Craig.McDaniel #18063: m68k struct alignment issue vs. PyException_HEAD http://bugs.python.org/issue18063 closed by pitrou #18067: In considering the current directory for importing modules, Py http://bugs.python.org/issue18067 closed by r.david.murray #18070: change importlib.util.module_for_loader to unconditionally set http://bugs.python.org/issue18070 closed by brett.cannon #18072: Add an implementation of importlib.abc.InspectLoader.get_code http://bugs.python.org/issue18072 closed by brett.cannon #18074: argparse: Namespace can contain critical characters http://bugs.python.org/issue18074 closed by Sworddragon #18075: Infinite recursion tests triggering a segfault http://bugs.python.org/issue18075 closed by lukasz.langa #18077: dis.dis raises IndexError http://bugs.python.org/issue18077 closed by serhiy.storchaka #18079: documentation tutorial 3.1.3 typo http://bugs.python.org/issue18079 closed by serhiy.storchaka #18080: setting CC no longer overrides default linker for extension mo http://bugs.python.org/issue18080 closed by ned.deily #18084: wave.py should use sys.byteorder to determine endianess http://bugs.python.org/issue18084 closed by serhiy.storchaka #18086: Create importlib.util.set_attrs_post_import() http://bugs.python.org/issue18086 closed by brett.cannon #18087: os.listdir don't show hidden option http://bugs.python.org/issue18087 closed by serhiy.storchaka #18097: spam http://bugs.python.org/issue18097 closed by amaury.forgeotdarc #18098: "Build Applet.app" build fails on OS X 10.8 http://bugs.python.org/issue18098 closed by ned.deily #18107: 'str(long)' can be made faster http://bugs.python.org/issue18107 closed by arigo #967161: pty.spawn() enhancements http://bugs.python.org/issue967161 closed by akuchling #706406: fix bug #685846: raw_input defers signals http://bugs.python.org/issue706406 closed by amaury.forgeotdarc #592703: HTTPS does not handle pipelined requests http://bugs.python.org/issue592703 closed by amaury.forgeotdarc #1554133: PyOS_InputHook() and related API funcs. not documented http://bugs.python.org/issue1554133 closed by akuchling

1 0

PEP 443 - request for pronouncement
by Łukasz Langa May 31, 2013

May 31, 2013

Hello python-dev, PEP 443 is ready for final review. I'm attaching the latest version below for convenience. The full history of changes is available here: http://hg.python.org/peps/log/tip/pep-0443.txt A reference implementation for PEP 443 is available at: http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363 with relevant tests here: http://hg.python.org/features/pep-443/file/tip/Lib/test/test_functools.py#l… and documentation here: http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l1… There's also an official backport for 2.6 - 3.3 already up: https://pypi.python.org/pypi/singledispatch PEP: 443 Title: Single-dispatch generic functions Version: $Revision$ Last-Modified: $Date$ Author: Łukasz Langa <lukasz(a)langa.pl> Discussions-To: Python-Dev <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 22-May-2013 Post-History: 22-May-2013, 25-May-2013, 31-May-2013 Replaces: 245, 246, 3124 Abstract ======== This PEP proposes a new mechanism in the ``functools`` standard library module that provides a simple form of generic programming known as single-dispatch generic functions. A **generic function** is composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as **single dispatch**. Rationale and Goals =================== Python has always provided a variety of built-in and standard-library generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``, ``copy.copy()``, and most of the functions in the ``operator`` module. However, it currently: 1. does not have a simple or straightforward way for developers to create new generic functions, 2. does not have a standard way for methods to be added to existing generic functions (i.e., some are added using registration functions, others require defining ``__special__`` methods, possibly by monkeypatching). In addition, it is currently a common anti-pattern for Python code to inspect the types of received arguments, in order to decide what to do with the objects. For example, code may wish to accept either an object of some type, or a sequence of objects of that type. Currently, the "obvious way" to do this is by type inspection, but this is brittle and closed to extension. Abstract Base Classes make it easier to discover present behaviour, but don't help adding new behaviour. A developer using an already-written library may be unable to change how their objects are treated by such code, especially if the objects they are using were created by a third party. Therefore, this PEP proposes a uniform API to address dynamic overloading using decorators. User API ======== To define a generic function, decorate it with the ``@singledispatch`` decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly:: >>> from functools import singledispatch >>> @singledispatch ... def fun(arg, verbose=False): ... if verbose: ... print("Let me just say,", end=" ") ... print(arg) To add overloaded implementations to the function, use the ``register()`` attribute of the generic function. It is a decorator, taking a type parameter and decorating a function implementing the operation for that type:: >>> @fun.register(int) ... def _(arg, verbose=False): ... if verbose: ... print("Strength in numbers, eh?", end=" ") ... print(arg) ... >>> @fun.register(list) ... def _(arg, verbose=False): ... if verbose: ... print("Enumerate this:") ... for i, elem in enumerate(arg): ... print(i, elem) To enable registering lambdas and pre-existing functions, the ``register()`` attribute can be used in a functional form:: >>> def nothing(arg, verbose=False): ... print("Nothing.") ... >>> fun.register(type(None), nothing) The ``register()`` attribute returns the undecorated function which enables decorator stacking, pickling, as well as creating unit tests for each variant independently:: >>> @fun.register(float) ... @fun.register(Decimal) ... def fun_num(arg, verbose=False): ... if verbose: ... print("Half of your number:", end=" ") ... print(arg / 2) ... >>> fun_num is fun False When called, the generic function dispatches on the type of the first argument:: >>> fun("Hello, world.") Hello, world. >>> fun("test.", verbose=True) Let me just say, test. >>> fun(42, verbose=True) Strength in numbers, eh? 42 >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True) Enumerate this: 0 spam 1 spam 2 eggs 3 spam >>> fun(None) Nothing. >>> fun(1.23) 0.615 Where there is no registered implementation for a specific type, its method resolution order is used to find a more generic implementation. To check which implementation will the generic function choose for a given type, use the ``dispatch()`` attribute:: >>> fun.dispatch(float) <function fun_num at 0x104319058> >>> fun.dispatch(dict) <function fun at 0x103fe4788> To access all registered implementations, use the read-only ``registry`` attribute:: >>> fun.registry.keys() dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>, <class 'decimal.Decimal'>, <class 'list'>, <class 'float'>]) >>> fun.registry[float] <function fun_num at 0x1035a2840> >>> fun.registry[object] <function fun at 0x103170788> The proposed API is intentionally limited and opinionated, as to ensure it is easy to explain and use, as well as to maintain consistency with existing members in the ``functools`` module. Implementation Notes ==================== The functionality described in this PEP is already implemented in the ``pkgutil`` standard library module as ``simplegeneric``. Because this implementation is mature, the goal is to move it largely as-is. The reference implementation is available on hg.python.org [#ref-impl]_. The dispatch type is specified as a decorator argument. An alternative form using function annotations has been considered but its inclusion has been deferred. As of May 2013, this usage pattern is out of scope for the standard library [#pep-0008]_ and the best practices for annotation usage are still debated. Based on the current ``pkgutil.simplegeneric`` implementation and following the convention on registering virtual subclasses on Abstract Base Classes, the dispatch registry will not be thread-safe. Abstract Base Classes --------------------- The ``pkgutil.simplegeneric`` implementation relied on several forms of method resultion order (MRO). ``@singledispatch`` removes special handling of old-style classes and Zope's ExtensionClasses. More importantly, it introduces support for Abstract Base Classes (ABC). When a generic function implementation is registered for an ABC, the dispatch algorithm switches to a mode of MRO calculation for the provided argument which includes the relevant ABCs. The algorithm is as follows:: def _compose_mro(cls, haystack): """Calculates the MRO for a given class `cls`, including relevant abstract base classes from `haystack`.""" bases = set(cls.__mro__) mro = list(cls.__mro__) for regcls in haystack: if regcls in bases or not issubclass(cls, regcls): continue # either present in the __mro__ or unrelated for index, base in enumerate(mro): if not issubclass(base, regcls): break if base in bases and not issubclass(regcls, base): # Conflict resolution: put classes present in __mro__ # and their subclasses first. index += 1 mro.insert(index, regcls) return mro In its most basic form, it returns the MRO for the given type:: >>> _compose_mro(dict, []) [<class 'dict'>, <class 'object'>] When the haystack consists of ABCs that the specified type is a subclass of, they are inserted in a predictable order:: >>> _compose_mro(dict, [Sized, MutableMapping, str, ... Sequence, Iterable]) [<class 'dict'>, <class 'collections.abc.MutableMapping'>, <class 'collections.abc.Iterable'>, <class 'collections.abc.Sized'>, <class 'object'>] While this mode of operation is significantly slower, all dispatch decisions are cached. The cache is invalidated on registering new implementations on the generic function or when user code calls ``register()`` on an ABC to register a new virtual subclass. In the latter case, it is possible to create a situation with ambiguous dispatch, for instance:: >>> from collections import Iterable, Container >>> class P: ... pass >>> Iterable.register(P) <class '__main__.P'> >>> Container.register(P) <class '__main__.P'> Faced with ambiguity, ``@singledispatch`` refuses the temptation to guess:: >>> @singledispatch ... def g(arg): ... return "base" ... >>> g.register(Iterable, lambda arg: "iterable") <function <lambda> at 0x108b49110> >>> g.register(Container, lambda arg: "container") <function <lambda> at 0x108b491c8> >>> g(P()) Traceback (most recent call last): ... RuntimeError: Ambiguous dispatch: <class 'collections.abc.Container'> or <class 'collections.abc.Iterable'> Note that this exception would not be raised if ``Iterable`` and ``Container`` had been provided as base classes during class definition. In this case dispatch happens in the MRO order:: >>> class Ten(Iterable, Container): ... def __iter__(self): ... for i in range(10): ... yield i ... def __contains__(self, value): ... return value in range(10) ... >>> g(Ten()) 'iterable' Usage Patterns ============== This PEP proposes extending behaviour only of functions specifically marked as generic. Just as a base class method may be overridden by a subclass, so too may a function be overloaded to provide custom functionality for a given type. Universal overloading does not equal *arbitrary* overloading, in the sense that we need not expect people to randomly redefine the behavior of existing functions in unpredictable ways. To the contrary, generic function usage in actual programs tends to follow very predictable patterns and registered implementations are highly-discoverable in the common case. If a module is defining a new generic operation, it will usually also define any required implementations for existing types in the same place. Likewise, if a module is defining a new type, then it will usually define implementations there for any generic functions that it knows or cares about. As a result, the vast majority of registered implementations can be found adjacent to either the function being overloaded, or to a newly-defined type for which the implementation is adding support. It is only in rather infrequent cases that one will have implementations registered in a module that contains neither the function nor the type(s) for which the implementation is added. In the absence of incompetence or deliberate intention to be obscure, the few implementations that are not registered adjacent to the relevant type(s) or function(s), will generally not need to be understood or known about outside the scope where those implementations are defined. (Except in the "support modules" case, where best practice suggests naming them accordingly.) As mentioned earlier, single-dispatch generics are already prolific throughout the standard library. A clean, standard way of doing them provides a way forward to refactor those custom implementations to use a common one, opening them up for user extensibility at the same time. Alternative approaches ====================== In PEP 3124 [#pep-3124]_ Phillip J. Eby proposes a full-grown solution with overloading based on arbitrary rule sets (with the default implementation dispatching on argument types), as well as interfaces, adaptation and method combining. PEAK-Rules [#peak-rules]_ is a reference implementation of the concepts described in PJE's PEP. Such a broad approach is inherently complex, which makes reaching a consensus hard. In contrast, this PEP focuses on a single piece of functionality that is simple to reason about. It's important to note this does not preclude the use of other approaches now or in the future. In a 2005 article on Artima [#artima2005]_ Guido van Rossum presents a generic function implementation that dispatches on types of all arguments on a function. The same approach was chosen in Andrey Popp's ``generic`` package available on PyPI [#pypi-generic]_, as well as David Mertz's ``gnosis.magic.multimethods`` [#gnosis-multimethods]_. While this seems desirable at first, I agree with Fredrik Lundh's comment that "if you design APIs with pages of logic just to sort out what code a function should execute, you should probably hand over the API design to someone else". In other words, the single argument approach proposed in this PEP is not only easier to implement but also clearly communicates that dispatching on a more complex state is an anti-pattern. It also has the virtue of corresponding directly with the familiar method dispatch mechanism in object oriented programming. The only difference is whether the custom implementation is associated more closely with the data (object-oriented methods) or the algorithm (single-dispatch overloading). PyPy's RPython offers ``extendabletype`` [#pairtype]_, a metaclass which enables classes to be externally extended. In combination with ``pairtype()`` and ``pair()`` factories, this offers a form of single-dispatch generics. Acknowledgements ================ Apart from Phillip J. Eby's work on PEP 3124 [#pep-3124]_ and PEAK-Rules, influences include Paul Moore's original issue [#issue-5135]_ that proposed exposing ``pkgutil.simplegeneric`` as part of the ``functools`` API, Guido van Rossum's article on multimethods [#artima2005]_, and discussions with Raymond Hettinger on a general pprint rewrite. Huge thanks to Nick Coghlan for encouraging me to create this PEP and providing initial feedback. References ========== .. [#ref-impl] http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359 .. [#pep-0008] PEP 8 states in the "Programming Recommendations" section that "the Python standard library will not use function annotations as that would result in a premature commitment to a particular annotation style". (http://www.python.org/dev/peps/pep-0008) .. [#pep-3124] http://www.python.org/dev/peps/pep-3124/ .. [#peak-rules] http://peak.telecommunity.com/DevCenter/PEAK_2dRules .. [#artima2005] http://www.artima.com/weblogs/viewpost.jsp?thread=101605 .. [#pypi-generic] http://pypi.python.org/pypi/generic .. [#gnosis-multimethods] http://gnosis.cx/publish/programming/charming_python_b12.html .. [#pairtype] https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py .. [#issue-5135] http://bugs.python.org/issue5135 Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev

2 3

PEP 443 - Single-dispatch generic functions (including ABC support)
by Łukasz Langa May 31, 2013

May 31, 2013

Hello, Since the initial version, several minor changes have been made to the PEP. The history is visible on hg.python.org. The most important change in this version is that I introduced ABC support and completed a reference implementation. No open issues remain from my point of view. PEP: 443 Title: Single-dispatch generic functions Version: $Revision$ Last-Modified: $Date$ Author: Łukasz Langa <lukasz(a)langa.pl> Discussions-To: Python-Dev <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 22-May-2013 Post-History: 22-May-2013, 25-May-2013 Replaces: 245, 246, 3124 Abstract ======== This PEP proposes a new mechanism in the ``functools`` standard library module that provides a simple form of generic programming known as single-dispatch generic functions. A **generic function** is composed of multiple functions sharing the same name. Which form should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as **single dispatch**. Rationale and Goals =================== Python has always provided a variety of built-in and standard-library generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``, ``copy.copy()``, and most of the functions in the ``operator`` module. However, it currently: 1. does not have a simple or straightforward way for developers to create new generic functions, 2. does not have a standard way for methods to be added to existing generic functions (i.e., some are added using registration functions, others require defining ``__special__`` methods, possibly by monkeypatching). In addition, it is currently a common anti-pattern for Python code to inspect the types of received arguments, in order to decide what to do with the objects. For example, code may wish to accept either an object of some type, or a sequence of objects of that type. Currently, the "obvious way" to do this is by type inspection, but this is brittle and closed to extension. Abstract Base Classes make it easier to discover present behaviour, but don't help adding new behaviour. A developer using an already-written library may be unable to change how their objects are treated by such code, especially if the objects they are using were created by a third party. Therefore, this PEP proposes a uniform API to address dynamic overloading using decorators. User API ======== To define a generic function, decorate it with the ``@singledispatch`` decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly:: >>> from functools import singledispatch >>> @singledispatch ... def fun(arg, verbose=False): ... if verbose: ... print("Let me just say,", end=" ") ... print(arg) To add overloaded implementations to the function, use the ``register()`` attribute of the generic function. It takes a type parameter:: >>> @fun.register(int) ... def _(arg, verbose=False): ... if verbose: ... print("Strength in numbers, eh?", end=" ") ... print(arg) ... >>> @fun.register(list) ... def _(arg, verbose=False): ... if verbose: ... print("Enumerate this:") ... for i, elem in enumerate(arg): ... print(i, elem) To enable registering lambdas and pre-existing functions, the ``register()`` attribute can be used in a functional form:: >>> def nothing(arg, verbose=False): ... print("Nothing.") ... >>> fun.register(type(None), nothing) The ``register()`` attribute returns the undecorated function which enables decorator stacking, pickling, as well as creating unit tests for each variant independently:: >>> @fun.register(float) ... @fun.register(Decimal) ... def fun_num(arg, verbose=False): ... if verbose: ... print("Half of your number:", end=" ") ... print(arg / 2) ... >>> fun_num is fun False When called, the generic function dispatches on the first argument:: >>> fun("Hello, world.") Hello, world. >>> fun("test.", verbose=True) Let me just say, test. >>> fun(42, verbose=True) Strength in numbers, eh? 42 >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True) Enumerate this: 0 spam 1 spam 2 eggs 3 spam >>> fun(None) Nothing. >>> fun(1.23) 0.615 To get the implementation for a specific type, use the ``dispatch()`` attribute:: >>> fun.dispatch(float) <function fun_num at 0x104319058> >>> fun.dispatch(dict) <function fun at 0x103fe4788> The proposed API is intentionally limited and opinionated, as to ensure it is easy to explain and use, as well as to maintain consistency with existing members in the ``functools`` module. Implementation Notes ==================== The functionality described in this PEP is already implemented in the ``pkgutil`` standard library module as ``simplegeneric``. Because this implementation is mature, the goal is to move it largely as-is. The reference implementation is available on hg.python.org [#ref-impl]_. The dispatch type is specified as a decorator argument. An alternative form using function annotations has been considered but its inclusion has been deferred. As of May 2013, this usage pattern is out of scope for the standard library [#pep-0008]_ and the best practices for annotation usage are still debated. Based on the current ``pkgutil.simplegeneric`` implementation and following the convention on registering virtual subclasses on Abstract Base Classes, the dispatch registry will not be thread-safe. Abstract Base Classes --------------------- The ``pkgutil.simplegeneric`` implementation relied on several forms of method resultion order (MRO). ``@singledispatch`` removes special handling of old-style classes and Zope's ExtensionClasses. More importantly, it introduces support for Abstract Base Classes (ABC). When a generic function overload is registered for an ABC, the dispatch algorithm switches to a mode of MRO calculation for the provided argument which includes the relevant ABCs. The algorithm is as follows:: def _compose_mro(cls, haystack): """Calculates the MRO for a given class `cls`, including relevant abstract base classes from `haystack`.""" bases = set(cls.__mro__) mro = list(cls.__mro__) for regcls in haystack: if regcls in bases or not issubclass(cls, regcls): continue # either present in the __mro__ or unrelated for index, base in enumerate(mro): if not issubclass(base, regcls): break if base in bases and not issubclass(regcls, base): # Conflict resolution: put classes present in __mro__ # and their subclasses first. index += 1 mro.insert(index, regcls) return mro While this mode of operation is significantly slower, no caching is involved because user code may ``register()`` a new class on an ABC at any time. In such case, it is possible to create a situation with ambiguous dispatch, for instance:: >>> from collections import Iterable, Container >>> class P: ... pass >>> Iterable.register(P) <class '__main__.P'> >>> Container.register(P) <class '__main__.P'> Faced with ambiguity, ``@singledispatch`` refuses the temptation to guess:: >>> @singledispatch ... def g(arg): ... return "base" ... >>> g.register(Iterable, lambda arg: "iterable") <function <lambda> at 0x108b49110> >>> g.register(Container, lambda arg: "container") <function <lambda> at 0x108b491c8> >>> g(P()) Traceback (most recent call last): ... RuntimeError: Ambiguous dispatch: <class 'collections.abc.Container'> or <class 'collections.abc.Iterable'> Note that this exception would not be raised if ``Iterable`` and ``Container`` had been provided as base classes during class definition. In this case dispatch happens in the MRO order:: >>> class Ten(Iterable, Container): ... def __iter__(self): ... for i in range(10): ... yield i ... def __contains__(self, value): ... return value in range(10) ... >>> g(Ten()) 'iterable' Usage Patterns ============== This PEP proposes extending behaviour only of functions specifically marked as generic. Just as a base class method may be overridden by a subclass, so too may a function be overloaded to provide custom functionality for a given type. Universal overloading does not equal *arbitrary* overloading, in the sense that we need not expect people to randomly redefine the behavior of existing functions in unpredictable ways. To the contrary, generic function usage in actual programs tends to follow very predictable patterns and overloads are highly-discoverable in the common case. If a module is defining a new generic operation, it will usually also define any required overloads for existing types in the same place. Likewise, if a module is defining a new type, then it will usually define overloads there for any generic functions that it knows or cares about. As a result, the vast majority of overloads can be found adjacent to either the function being overloaded, or to a newly-defined type for which the overload is adding support. It is only in rather infrequent cases that one will have overloads in a module that contains neither the function nor the type(s) for which the overload is added. In the absence of incompetence or deliberate intention to be obscure, the few overloads that are not adjacent to the relevant type(s) or function(s), will generally not need to be understood or known about outside the scope where those overloads are defined. (Except in the "support modules" case, where best practice suggests naming them accordingly.) As mentioned earlier, single-dispatch generics are already prolific throughout the standard library. A clean, standard way of doing them provides a way forward to refactor those custom implementations to use a common one, opening them up for user extensibility at the same time. Alternative approaches ====================== In PEP 3124 [#pep-3124]_ Phillip J. Eby proposes a full-grown solution with overloading based on arbitrary rule sets (with the default implementation dispatching on argument types), as well as interfaces, adaptation and method combining. PEAK-Rules [#peak-rules]_ is a reference implementation of the concepts described in PJE's PEP. Such a broad approach is inherently complex, which makes reaching a consensus hard. In contrast, this PEP focuses on a single piece of functionality that is simple to reason about. It's important to note this does not preclude the use of other approaches now or in the future. In a 2005 article on Artima [#artima2005]_ Guido van Rossum presents a generic function implementation that dispatches on types of all arguments on a function. The same approach was chosen in Andrey Popp's ``generic`` package available on PyPI [#pypi-generic]_, as well as David Mertz's ``gnosis.magic.multimethods`` [#gnosis-multimethods]_. While this seems desirable at first, I agree with Fredrik Lundh's comment that "if you design APIs with pages of logic just to sort out what code a function should execute, you should probably hand over the API design to someone else". In other words, the single argument approach proposed in this PEP is not only easier to implement but also clearly communicates that dispatching on a more complex state is an anti-pattern. It also has the virtue of corresponding directly with the familiar method dispatch mechanism in object oriented programming. The only difference is whether the custom implementation is associated more closely with the data (object-oriented methods) or the algorithm (single-dispatch overloading). PyPy's RPython offers ``extendabletype`` [#pairtype]_, a metaclass which enables classes to be externally extended. In combination with ``pairtype()`` and ``pair()`` factories, this offers a form of single-dispatch generics. Acknowledgements ================ Apart from Phillip J. Eby's work on PEP 3124 [#pep-3124]_ and PEAK-Rules, influences include Paul Moore's original issue [#issue-5135]_ that proposed exposing ``pkgutil.simplegeneric`` as part of the ``functools`` API, Guido van Rossum's article on multimethods [#artima2005]_, and discussions with Raymond Hettinger on a general pprint rewrite. Huge thanks to Nick Coghlan for encouraging me to create this PEP and providing initial feedback. References ========== .. [#ref-impl] http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359 .. [#pep-0008] PEP 8 states in the "Programming Recommendations" section that "the Python standard library will not use function annotations as that would result in a premature commitment to a particular annotation style". (http://www.python.org/dev/peps/pep-0008) .. [#pep-3124] http://www.python.org/dev/peps/pep-3124/ .. [#peak-rules] http://peak.telecommunity.com/DevCenter/PEAK_2dRules .. [#artima2005] http://www.artima.com/weblogs/viewpost.jsp?thread=101605 .. [#pypi-generic] http://pypi.python.org/pypi/generic .. [#gnosis-multimethods] http://gnosis.cx/publish/programming/charming_python_b12.html .. [#pairtype] https://bitbucket.org/pypy/pypy/raw/default/rpython/tool/pairtype.py .. [#issue-5135] http://bugs.python.org/issue5135 Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev

7 28

performance testing recommendations in devguide
by Eric Snow May 30, 2013

May 30, 2013

The devguide doesn't have anything on performance testing that I could find. We do have a number of relatively useful resources in this space though, like pybench and (eventually) speed.python.org. I'd like to add a page to the devguide on performance testing, including an explanation of our performance goals, how to test for them, and what tools are available. Tools I'm aware of: * pybench (relatively limited in real-world usefulness) * timeit module (for quick comparisions) * benchmarks repo (real-world performance test suite) * speed.python.org (would omit for now) Things to test: * speed * memory (tools? tests?) Critically sensitive performance subjects * interpreter start-up time * module import overhead * attribute lookup overhead (including MRO traversal) * function call overhead * instance creation overhead * dict performance (the underlying namespace type) * tuple performance (packing/unpacking, integral container type) * string performance What would be important to say in the devguide regarding Python performance and testing it? What would you add/subtract from the above? How important is testing memory performance? How do we avoid performance regressions? Thanks! -eric

7 7

Segmentation fault on 3.4 with --pydebug
by Łukasz Langa May 30, 2013

May 30, 2013

This happens after Benjamin's changes in 83937. Anybody else seeing this? Intel i5 2.4 GHz, Mac OS X 10.8.3, clang $ hg up default $ make distclean $ MACOSX_DEPLOYMENT_TARGET=10.8 ./configure --with-pydebug $ make $ ./python.exe -Wd -m test.regrtest test_exceptions [1/1] test_exceptions Fatal Python error: Segmentation fault Current thread 0x00007fff74254180: File "/Users/ambv/Documents/Projekty/Python/cpython/py34/Lib/test/test_exceptions.py", line 453 in f File "/Users/ambv/Documents/Projekty/Python/cpython/py34/Lib/test/test_exceptions.py", line 453 in f File "/Users/ambv/Documents/Projekty/Python/cpython/py34/Lib/test/test_exceptions.py", line 453 in f ... (repeated a 100 times) Command terminated abnormally. Everything runs fine without --with-pydebug (or before 83937 with --with-pydebug). -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev

5 5

Re: [Python-Dev] cpython: Introduce importlib.util.ModuleManager which is a context manager to
by Antoine Pitrou May 30, 2013

May 30, 2013

On Tue, 28 May 2013 23:29:46 +0200 (CEST) brett.cannon <python-checkins(a)python.org> wrote: > > +.. class:: ModuleManager(name) > + > + A :term:`context manager` which provides the module to load. The module will > + either come from :attr:`sys.modules` in the case of reloading or a fresh > + module if loading a new module. Proper cleanup of :attr:`sys.modules` occurs > + if the module was new and an exception was raised. What use case does this API solve? (FWIW, I think "ModuleManager" is a rather bad name :-)) Regards Antoine.

8 21

Bilingual scripts
by Barry Warsaw May 30, 2013

May 30, 2013

Here's something that seems to come up from time to time in Debian. Take a Python application like tox, nose, or pyflakes. Their executables work with both Python 2 and 3, but require a #! line to choose which interpreter to invoke. When we add Python 3 support in Debian for such a script, all the library installations are handled just fine, but we have conflicts about what to name the thing that lands in /usr/bin. Do we have /usr/bin/pyflakes and /usr/bin/pyflakes3? Do we call the latter py3flakes (as has been convention with some other scripts, but which breaks locate(1))? Do we simply remove the /usr/bin scripts and encourage people to use something like `$python -m nose`? One interesting recent suggestion is to create a shell driver script and make that thing accept --py3 and --py2 flags, which would then select the appropriate interpreter and invoke the actual Python driver. There are some technical details to iron out with that, but there's some appeal to it. Have any other *nix distros addressed this, and if so, how do you solve it? It would be nice if we could have some cross-platform recommendations so things work the same wherever you go. To that end, if we can reach some consensus, I'd be willing to put together an informational PEP and some scripts that might be of general use. -Barry

9 29

Structural cleanups to the main CPython repo
by Nick Coghlan May 29, 2013

May 29, 2013

I have a feature branch where I'm intermittently working on the bootstrapping changes described in PEP 432. As part of those changes, I've cleaned up a few aspects of the repo layout: * moved the main executable source file from Modules to a separate Apps directory * moved the _freezeimportlib and _testembed source files from Modules to a separate Tools directory * split the monster pythonrun.h/c pair into 3 separate header/impl pairs: * bootstrap.h/bootstrap.c * shutdown.h/shutdown.c * pythonrun.h/pythonrun.c These structural changes generally mean automatic merges touching the build machinery or the startup or shutdown code fail fairly spectacularly and need a lot of TLC to complete them without losing any changes from the main repo. Would anyone object if I went ahead and posted patches for making these changes to the main repo? I found they made the code *much* easier to follow when I started to turn the ideas in PEP 432 into working software, and implementing these shifts should make future merges to my feature branch simpler, as well as producing significantly cleaner diffs when PEP 432 gets closer to completion. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

8 13

Re: [Python-Dev] cpython (merge 3.3 -> default): Merge with 3.3
by Antoine Pitrou May 29, 2013

May 29, 2013

On Tue, 28 May 2013 15:06:39 -0400 Terry Reedy <tjreedy(a)udel.edu> wrote: > On 5/28/2013 2:14 AM, Benjamin Peterson wrote: > > 2013/5/27 terry.reedy <python-checkins(a)python.org>: > >> http://hg.python.org/cpython/rev/c5d4c041ab47 > >> changeset: 83942:c5d4c041ab47 > >> parent: 83940:2ea849fde22b > >> parent: 83941:24c3e7e08168 > >> user: Terry Jan Reedy <tjreedy(a)udel.edu> > >> date: Mon May 27 21:33:40 2013 -0400 > >> summary: > >> Merge with 3.3 > >> > >> files: > >> Lib/idlelib/CallTips.py | 4 +- > >> Lib/idlelib/PathBrowser.py | 3 +- > >> Lib/idlelib/idle_test/(a)README.txt | 63 +++++++++++ > > Is @README really the intended name of this file? > Yes, Nick suggested README instead of what I had. I want a prefix to > keep it near the top of a directory listing even when other non > 'test_xxx' files are added. I thing '_' wold be better though. I don't think "prefixing with a weird character so that the filename show up top" is a very elegant trick, and we don't use it for other directories. "README.txt" *will* be easily visible because of the all-caps filename. > If I can > find how to rename in hg "Rename in hg" -> "hg rename" Antoine.

3 3

pep 422 "Safe object finalization" question: why break weakrefs first?
by Russell E. Owen May 28, 2013

May 28, 2013

Pep 422 proposes the following order for dealing with cyclic isolates: 1. Weakrefs to CI objects are cleared, and their callbacks called. At this point, the objects are still safe to use. 2. The finalizers of all CI objects are called. 3. The CI is traversed again to determine if it is still isolated. If it is determined that at least one object in CI is now reachable from outside the CI, this collection is aborted and the whole CI is resurrected. Otherwise, proceed. 4. The CI becomes a CT as the GC systematically breaks all known references inside it (using the tp_clear function). Why are weakrefs are broken first, before determining if any of the objects should become resurrected? Naively I would expect weakrefs to be broken after step 3, once the system is sure no objects have been resurrected. I request that this information be added to PEP 422. -- Russell

2 1

PEP 8 and function names
by Sébastien Durand May 28, 2013

May 28, 2013

Hi all, "There should be one-- and preferably only one --obvious way to do it." We all love this mantra. But one thing that often confuses people : function naming. The standard library is kind of inconsistent. Some functions are separated by underscores and others aren't. It's not intuitive and new pythonistas end up constantly reading the doc. (Time saving one char typing vs time guessing function names.) Would it be a good idea to clarify PEP 8 on this ? I mean for future libraries. Regards, Sébastien

7 6

Re: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Merge with 3.3
by Benjamin Peterson May 28, 2013

May 28, 2013

2013/5/27 terry.reedy <python-checkins(a)python.org>: > http://hg.python.org/cpython/rev/c5d4c041ab47 > changeset: 83942:c5d4c041ab47 > parent: 83940:2ea849fde22b > parent: 83941:24c3e7e08168 > user: Terry Jan Reedy <tjreedy(a)udel.edu> > date: Mon May 27 21:33:40 2013 -0400 > summary: > Merge with 3.3 > > files: > Lib/idlelib/CallTips.py | 4 +- > Lib/idlelib/PathBrowser.py | 3 +- > Lib/idlelib/idle_test/(a)README.txt | 63 +++++++++++ Is @README really the intended name of this file? Would README-TEST or something similar be better? -- Regards, Benjamin

1 0

__subclasses__() return order
by Antoine Pitrou May 26, 2013

May 26, 2013

Hello, In http://bugs.python.org/issue17936, I proposed making tp_subclasses (the internal container implementing object.__subclasses__) a dict. This would make the return order of __subclasses__ completely undefined, while it is right now slightly predictable. I have never seen __subclasses__ actually used in production code, so I'm wondering whether someone might be affected by such a change. Regards Antoine.

7 8

Summary of Python tracker Issues
by Python tracker May 24, 2013

May 24, 2013

ACTIVITY SUMMARY (2013-05-17 - 2013-05-24) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3972 ( +6) closed 25850 (+45) total 29822 (+51) Open issues with patches: 1773 Issues opened (41) ================== #13146: Writing a pyc file is not atomic http://bugs.python.org/issue13146 reopened by barry #15535: Fix pickling efficiency of named tuples in 2.7.3 http://bugs.python.org/issue15535 reopened by rhettinger #18000: _md5 should be built if _ssl cannot be built http://bugs.python.org/issue18000 opened by jdemeyer #18003: New lzma crazy slow with line-oriented reading. http://bugs.python.org/issue18003 opened by Michael.Fox #18004: test_list.test_overflow crashes Win64 http://bugs.python.org/issue18004 opened by anselm.kruis #18009: os.write.__doc__ is misleading http://bugs.python.org/issue18009 opened by shai #18010: pydoc search chokes on import errors http://bugs.python.org/issue18010 opened by pitrou #18011: Inconsistency between b32decode() documentation, docstring and http://bugs.python.org/issue18011 opened by serhiy.storchaka #18013: cgi.FieldStorage does not parse W3C sample http://bugs.python.org/issue18013 opened by flox #18014: Problem compiling on Windows, VC++Express 2010 http://bugs.python.org/issue18014 opened by terry.reedy #18015: python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or http://bugs.python.org/issue18015 opened by anselm.kruis #18016: subprocess should open stdin in mode w+b on windows http://bugs.python.org/issue18016 opened by Jason.Gross #18017: ctypes.PyDLL documentation http://bugs.python.org/issue18017 opened by Marc.Brünink #18018: SystemError: Parent module '' not loaded, cannot perform relat http://bugs.python.org/issue18018 opened by flox #18020: html.escape 10x slower than cgi.escape http://bugs.python.org/issue18020 opened by flox #18021: Update broken link to Apple Publication Style Guide http://bugs.python.org/issue18021 opened by madison.may #18022: Inconsistency between quopri.decodestring() and email.quoprimi http://bugs.python.org/issue18022 opened by serhiy.storchaka #18023: msi product code for 2.7.5150 not in Tools/msi/uuids.py http://bugs.python.org/issue18023 opened by anselm.kruis #18024: dbm module fails to build on SLES11SP1 using 2.7.5 source http://bugs.python.org/issue18024 opened by wempa #18025: Buffer overflow in BufferedIOBase.readinto() http://bugs.python.org/issue18025 opened by serhiy.storchaka #18027: distutils should access stat_result timestamps via .st_*time a http://bugs.python.org/issue18027 opened by jwilk #18028: Warnings with -fstrict-aliasing http://bugs.python.org/issue18028 opened by bkabrda #18029: Python SSL support is missing from SPARC build http://bugs.python.org/issue18029 opened by eeiddne #18032: set methods should specify whether they consume iterators "laz http://bugs.python.org/issue18032 opened by hhm #18033: Example for Profile Module shows incorrect method http://bugs.python.org/issue18033 opened by jough #18034: Last two entries in the programming FAQ are out of date (impor http://bugs.python.org/issue18034 opened by r.david.murray #18035: telnetlib incorrectly assumes that select.error has an errno a http://bugs.python.org/issue18035 opened by gregory.p.smith #18036: "How do I create a .pyc file?" FAQ entry is out of date http://bugs.python.org/issue18036 opened by r.david.murray #18037: 2to3 passes through string literal which causes SyntaxError in http://bugs.python.org/issue18037 opened by vinay.sajip #18038: Unhelpful error message on invalid encoding specification http://bugs.python.org/issue18038 opened by Max.Cantor #18039: dbm.open(..., flag="n") does not work and does not give a warn http://bugs.python.org/issue18039 opened by sonyachiko #18040: SIGINT catching regression on windows in 2.7 http://bugs.python.org/issue18040 opened by David.Gilman #18041: mention issues with code churn in the devguide http://bugs.python.org/issue18041 opened by tshepang #18042: Provide enum.unique class decorator http://bugs.python.org/issue18042 opened by ncoghlan #18043: No mention of `match.regs` in `re` documentation http://bugs.python.org/issue18043 opened by cool-RR #18044: Email headers do not properly decode to unicode. http://bugs.python.org/issue18044 opened by Tim.Rawlinson #18045: get_python_version is not import in bdist_rpm.py http://bugs.python.org/issue18045 opened by maoliping455 #18046: Simplify and clarify logging internals http://bugs.python.org/issue18046 opened by alex #18047: Descriptors get invoked in old-style objects and classes http://bugs.python.org/issue18047 opened by icecrime #18048: Merging test_pep263.py and test_coding.py http://bugs.python.org/issue18048 opened by serhiy.storchaka #18049: Re-enable threading test on OSX http://bugs.python.org/issue18049 opened by ronaldoussoren Most recent 15 issues with no replies (15) ========================================== #18049: Re-enable threading test on OSX http://bugs.python.org/issue18049 #18047: Descriptors get invoked in old-style objects and classes http://bugs.python.org/issue18047 #18046: Simplify and clarify logging internals http://bugs.python.org/issue18046 #18045: get_python_version is not import in bdist_rpm.py http://bugs.python.org/issue18045 #18044: Email headers do not properly decode to unicode. http://bugs.python.org/issue18044 #18043: No mention of `match.regs` in `re` documentation http://bugs.python.org/issue18043 #18040: SIGINT catching regression on windows in 2.7 http://bugs.python.org/issue18040 #18039: dbm.open(..., flag="n") does not work and does not give a warn http://bugs.python.org/issue18039 #18036: "How do I create a .pyc file?" FAQ entry is out of date http://bugs.python.org/issue18036 #18034: Last two entries in the programming FAQ are out of date (impor http://bugs.python.org/issue18034 #18033: Example for Profile Module shows incorrect method http://bugs.python.org/issue18033 #18032: set methods should specify whether they consume iterators "laz http://bugs.python.org/issue18032 #18027: distutils should access stat_result timestamps via .st_*time a http://bugs.python.org/issue18027 #18025: Buffer overflow in BufferedIOBase.readinto() http://bugs.python.org/issue18025 #18023: msi product code for 2.7.5150 not in Tools/msi/uuids.py http://bugs.python.org/issue18023 Most recent 15 issues waiting for review (15) ============================================= #18049: Re-enable threading test on OSX http://bugs.python.org/issue18049 #18046: Simplify and clarify logging internals http://bugs.python.org/issue18046 #18038: Unhelpful error message on invalid encoding specification http://bugs.python.org/issue18038 #18025: Buffer overflow in BufferedIOBase.readinto() http://bugs.python.org/issue18025 #18020: html.escape 10x slower than cgi.escape http://bugs.python.org/issue18020 #18015: python 2.7.5 fails to unpickle namedtuple pickled by 2.7.3 or http://bugs.python.org/issue18015 #18013: cgi.FieldStorage does not parse W3C sample http://bugs.python.org/issue18013 #18011: Inconsistency between b32decode() documentation, docstring and http://bugs.python.org/issue18011 #18010: pydoc search chokes on import errors http://bugs.python.org/issue18010 #18000: _md5 should be built if _ssl cannot be built http://bugs.python.org/issue18000 #17998: internal error in regular expression engine http://bugs.python.org/issue17998 #17978: Python crashes if Py_Initialize/Py_Finalize are called multipl http://bugs.python.org/issue17978 #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 #17956: add ScheduledExecutor http://bugs.python.org/issue17956 Top 10 most discussed issues (10) ================================= #18003: New lzma crazy slow with line-oriented reading. http://bugs.python.org/issue18003 17 msgs #13612: xml.etree.ElementTree says unknown encoding of a regular encod http://bugs.python.org/issue13612 14 msgs #12641: Remove -mno-cygwin from distutils http://bugs.python.org/issue12641 13 msgs #15392: Create a unittest framework for IDLE http://bugs.python.org/issue15392 11 msgs #13146: Writing a pyc file is not atomic http://bugs.python.org/issue13146 9 msgs #17140: Provide a more obvious public ThreadPool API http://bugs.python.org/issue17140 9 msgs #7727: xmlrpc library returns string which contain null ( \x00 ) http://bugs.python.org/issue7727 7 msgs #14097: Improve the "introduction" page of the tutorial http://bugs.python.org/issue14097 7 msgs #17683: socket.getsockname() inconsistent return type with AF_UNIX http://bugs.python.org/issue17683 7 msgs #17839: base64 module should use memoryview http://bugs.python.org/issue17839 7 msgs Issues closed (42) ================== #3006: subprocess.Popen causes socket to remain open after close http://bugs.python.org/issue3006 closed by neologix #3489: add rotate{left,right} methods to bytearray http://bugs.python.org/issue3489 closed by terry.reedy #7146: platform.uname()[4] returns 'amd64' on Windows and 'x86-64' on http://bugs.python.org/issue7146 closed by r.david.murray #7214: TreeBuilder.end(tag) differs between cElementTree and ElementT http://bugs.python.org/issue7214 closed by eli.bendersky #11011: More functools functions http://bugs.python.org/issue11011 closed by r.david.murray #11995: test_pydoc loads all Python modules http://bugs.python.org/issue11995 closed by pitrou #14009: Clearer documentation for cElementTree http://bugs.python.org/issue14009 closed by eli.bendersky #15758: FileIO.readall() has worst case O(n^2) complexity http://bugs.python.org/issue15758 closed by sbt #16603: Sporadic test_socket failures: testFDPassCMSG_SPACE on Mac OS http://bugs.python.org/issue16603 closed by neologix #16986: ElementTree incorrectly parses strings with declared encoding http://bugs.python.org/issue16986 closed by serhiy.storchaka #17269: getaddrinfo segfaults on OS X when provided with invalid argum http://bugs.python.org/issue17269 closed by ronaldoussoren #17453: logging.config.fileConfig error http://bugs.python.org/issue17453 closed by lukasz.langa #17532: IDLE: Always include "Options" menu on MacOSX http://bugs.python.org/issue17532 closed by ned.deily #17644: str.format() crashes http://bugs.python.org/issue17644 closed by python-dev #17684: Skip tests in test_socket like testFDPassSeparate on OS X http://bugs.python.org/issue17684 closed by neologix #17743: Use extended syntax of `set` command in activate.bat/deactivat http://bugs.python.org/issue17743 closed by python-dev #17744: Unset VIRTUAL_ENV environment variable in deactivate.bat http://bugs.python.org/issue17744 closed by python-dev #17812: Quadratic complexity in b32encode http://bugs.python.org/issue17812 closed by serhiy.storchaka #17844: Add link to alternatives for bytes-to-bytes codecs http://bugs.python.org/issue17844 closed by ncoghlan #17900: Recursive OrderedDict pickling http://bugs.python.org/issue17900 closed by serhiy.storchaka #17901: _elementtree.TreeBuilder raises IndexError on end if construct http://bugs.python.org/issue17901 closed by eli.bendersky #17917: use PyModule_AddIntMacro() instead of PyModule_AddIntConstant( http://bugs.python.org/issue17917 closed by neologix #17937: Collect garbage harder at shutdown http://bugs.python.org/issue17937 closed by pitrou #17955: Minor updates to Functional HOWTO http://bugs.python.org/issue17955 closed by akuchling #17957: remove outdated (and unexcellent) paragraph in whatsnew http://bugs.python.org/issue17957 closed by ezio.melotti #17979: Cannot build 2.7 with --enable-unicode=no http://bugs.python.org/issue17979 closed by serhiy.storchaka #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 closed by pitrou #17988: ElementTree.Element != ElementTree._ElementInterface http://bugs.python.org/issue17988 closed by eli.bendersky #17989: ElementTree.Element broken attribute setting http://bugs.python.org/issue17989 closed by eli.bendersky #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 closed by giampaolo.rodola #17999: test_super fails in refleak runs http://bugs.python.org/issue17999 closed by python-dev #18001: TypeError: dict is not callable in ConfigParser.py http://bugs.python.org/issue18001 closed by ethan.furman #18002: AMD64 Windows7 SP1 3.x buildbot: compilation of _ssl module fa http://bugs.python.org/issue18002 closed by pitrou #18005: faster modular exponentiation in some cases http://bugs.python.org/issue18005 closed by mark.dickinson #18006: Set thread name in linux kernel http://bugs.python.org/issue18006 closed by neologix #18007: CookieJar expects request objects with origin_req_host attribu http://bugs.python.org/issue18007 closed by orsenthil #18008: python33-3.3.2 Parser/pgen: Permission denied http://bugs.python.org/issue18008 closed by ned.deily #18012: cannot assign unicode keys to SimpleCookie http://bugs.python.org/issue18012 closed by flox #18019: dictionary views lead to segmentation fault http://bugs.python.org/issue18019 closed by python-dev #18030: IDLE shell crashes when reporting errors in Windows 7 http://bugs.python.org/issue18030 closed by serhiy.storchaka #18031: The Python Tutorial says % string formatting will be removed http://bugs.python.org/issue18031 closed by rhettinger #18026: Typo in ctypes documentation http://bugs.python.org/issue18026 closed by ned.deily

1 0

Re: [Python-Dev] PEP 443 - Single-dispatch generic functions
by Łukasz Langa May 24, 2013

May 24, 2013

On 24 maj 2013, at 14:53, Ronan Lamy <ronan.lamy(a)gmail.com> wrote: >> 2013/5/24 Łukasz Langa <lukasz(a)langa.pl> >> >> I recognize the need for such behaviour to be discoverable. This is >> important for debugging purposes. This is why I'm going to let users >> inspect registered overloads, as well as provide their own mapping >> for the registry. I'm working on the reference implementation now, >> stay tuned. > > OK, I agree that preventing silent overwriting is actually not desirable. Specifying an interface for the implementation registry sounds like the best possible solution to my concerns. Users can now inspect the dispatcher and specify their own mapping for a registry: http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l359 Documentation: http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l1… ABC support is still not there, I'm working on it. -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev

1 0

PEP 443 - Single-dispatch generic functions
by Łukasz Langa May 24, 2013

May 24, 2013

Hello, I would like to submit the following PEP for discussion and evaluation. PEP: 443 Title: Single-dispatch generic functions Version: $Revision$ Last-Modified: $Date$ Author: Łukasz Langa <lukasz(a)langa.pl> Discussions-To: Python-Dev <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 22-May-2013 Post-History: 22-May-2013 Replaces: 245, 246, 3124 Abstract ======== This PEP proposes a new mechanism in the ``functools`` standard library module that provides a simple form of generic programming known as single-dispatch generic functions. A **generic function** is composed of multiple functions sharing the same name. Which form should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as **single dispatch**. Rationale and Goals =================== Python has always provided a variety of built-in and standard-library generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``, ``copy.copy()``, and most of the functions in the ``operator`` module. However, it currently: 1. does not have a simple or straightforward way for developers to create new generic functions, 2. does not have a standard way for methods to be added to existing generic functions (i.e., some are added using registration functions, others require defining ``__special__`` methods, possibly by monkeypatching). In addition, it is currently a common anti-pattern for Python code to inspect the types of received arguments, in order to decide what to do with the objects. For example, code may wish to accept either an object of some type, or a sequence of objects of that type. Currently, the "obvious way" to do this is by type inspection, but this is brittle and closed to extension. Abstract Base Classes make it easier to discover present behaviour, but don't help adding new behaviour. A developer using an already-written library may be unable to change how their objects are treated by such code, especially if the objects they are using were created by a third party. Therefore, this PEP proposes a uniform API to address dynamic overloading using decorators. User API ======== To define a generic function, decorate it with the ``@singledispatch`` decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly: .. code-block:: pycon >>> from functools import singledispatch >>> @singledispatch ... def fun(arg, verbose=False): ... if verbose: ... print("Let me just say,", end=" ") ... print(arg) To add overloaded implementations to the function, use the ``register()`` attribute of the generic function. It takes a type parameter: .. code-block:: pycon >>> @fun.register(int) ... def _(arg, verbose=False): ... if verbose: ... print("Strength in numbers, eh?", end=" ") ... print(arg) ... >>> @fun.register(list) ... def _(arg, verbose=False): ... if verbose: ... print("Enumerate this:") ... for i, elem in enumerate(arg): ... print(i, elem) To enable registering lambdas and pre-existing functions, the ``register()`` attribute can be used in a functional form: .. code-block:: pycon >>> def nothing(arg, verbose=False): ... print("Nothing.") ... >>> fun.register(type(None), nothing) When called, the function dispatches on the first argument: .. code-block:: pycon >>> fun("Hello, world.") Hello, world. >>> fun("test.", verbose=True) Let me just say, test. >>> fun(42, verbose=True) Strength in numbers, eh? 42 >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True) Enumerate this: 0 spam 1 spam 2 eggs 3 spam >>> fun(None) Nothing. The proposed API is intentionally limited and opinionated, as to ensure it is easy to explain and use, as well as to maintain consistency with existing members in the ``functools`` module. Implementation Notes ==================== The functionality described in this PEP is already implemented in the ``pkgutil`` standard library module as ``simplegeneric``. Because this implementation is mature, the goal is to move it largely as-is. Several open issues remain: * the current implementation relies on ``__mro__`` alone, making it incompatible with Abstract Base Classes' ``register()``/``unregister()`` functionality. A possible solution has been proposed by PJE on the original issue for exposing ``pkgutil.simplegeneric`` as part of the ``functools`` API [#issue-5135]_. * the dispatch type is currently specified as a decorator argument. The implementation could allow a form using argument annotations. This usage pattern is out of scope for the standard library [#pep-0008]_. However, whether this registration form would be acceptable for general usage, is up to debate. Based on the current ``pkgutil.simplegeneric`` implementation and following the convention on registering virtual subclasses on Abstract Base Classes, the dispatch registry will not be thread-safe. Usage Patterns ============== This PEP proposes extending behaviour only of functions specifically marked as generic. Just as a base class method may be overridden by a subclass, so too may a function be overloaded to provide custom functionality for a given type. Universal overloading does not equal *arbitrary* overloading, in the sense that we need not expect people to randomly redefine the behavior of existing functions in unpredictable ways. To the contrary, generic function usage in actual programs tends to follow very predictable patterns and overloads are highly-discoverable in the common case. If a module is defining a new generic operation, it will usually also define any required overloads for existing types in the same place. Likewise, if a module is defining a new type, then it will usually define overloads there for any generic functions that it knows or cares about. As a result, the vast majority of overloads can be found adjacent to either the function being overloaded, or to a newly-defined type for which the overload is adding support. It is only in rather infrequent cases that one will have overloads in a module that contains neither the function nor the type(s) for which the overload is added. In the absence of incompetence or deliberate intention to be obscure, the few overloads that are not adjacent to the relevant type(s) or function(s), will generally not need to be understood or known about outside the scope where those overloads are defined. (Except in the "support modules" case, where best practice suggests naming them accordingly.) As mentioned earlier, single-dispatch generics are already prolific throughout the standard library. A clean, standard way of doing them provides a way forward to refactor those custom implementations to use a common one, opening them up for user extensibility at the same time. Alternative approaches ====================== In PEP 3124 [#pep-3124]_ Phillip J. Eby proposes a full-grown solution with overloading based on arbitrary rule sets (with the default implementation dispatching on argument types), as well as interfaces, adaptation and method combining. PEAK-Rules [#peak-rules]_ is a reference implementation of the concepts described in PJE's PEP. Such a broad approach is inherently complex, which makes reaching a consensus hard. In contrast, this PEP focuses on a single piece of functionality that is simple to reason about. It's important to note this does not preclude the use of other approaches now or in the future. In a 2005 article on Artima [#artima2005]_ Guido van Rossum presents a generic function implementation that dispatches on types of all arguments on a function. The same approach was chosen in Andrey Popp's ``generic`` package available on PyPI [#pypi-generic]_, as well as David Mertz's ``gnosis.magic.multimethods`` [#gnosis-multimethods]_. While this seems desirable at first, I agree with Fredrik Lundh's comment that "if you design APIs with pages of logic just to sort out what code a function should execute, you should probably hand over the API design to someone else". In other words, the single argument approach proposed in this PEP is not only easier to implement but also clearly communicates that dispatching on a more complex state is an anti-pattern. It also has the virtue of corresponding directly with the familiar method dispatch mechanism in object oriented programming. The only difference is whether the custom implementation is associated more closely with the data (object-oriented methods) or the algorithm (single-dispatch overloading). Acknowledgements ================ Apart from Phillip J. Eby's work on PEP 3124 [#pep-3124]_ and PEAK-Rules, influences include Paul Moore's original issue [#issue-5135]_ that proposed exposing ``pkgutil.simplegeneric`` as part of the ``functools`` API, Guido van Rossum's article on multimethods [#artima2005]_, and discussions with Raymond Hettinger on a general pprint rewrite. References ========== .. [#issue-5135] http://bugs.python.org/issue5135 .. [#pep-0008] PEP 8 states in the "Programming Recommendations" section that "the Python standard library will not use function annotations as that would result in a premature commitment to a particular annotation style". (http://www.python.org/dev/peps/pep-0008) .. [#pep-3124] http://www.python.org/dev/peps/pep-3124/ .. [#peak-rules] http://peak.telecommunity.com/DevCenter/PEAK_2dRules .. [#artima2005] http://www.artima.com/weblogs/viewpost.jsp?thread=101605 .. [#pypi-generic] http://pypi.python.org/pypi/generic .. [#gnosis-multimethods] http://gnosis.cx/publish/programming/charming_python_b12.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev

18 52

What if we didn't have repr?
by Guido van Rossum May 24, 2013

May 24, 2013

On Sun, May 19, 2013 at 4:27 PM, Gregory P. Smith <greg(a)krypto.org> wrote: > Now you've got me wondering what Python would be like if repr, `` and > __repr__ never existed as language features. Upon first thoughts, I actually > don't see much downside (no, i'm not advocating making that change). > Something to ponder. I have pondered it many times, although usually in the form "Why do we need both str and repr?" Unfortunately I always come back to the same issue: I really want print() of a string to write just the characters of the string (without quotes), but I also really want the >>> prompt to write the string with quotes (and escapes). Moreover, these are just two examples of the different use cases -- repr() is more useful whenever you are writing a value for a debugging purpose (e.g. when logging), and str() is more useful when writing a value as "output" of a program. One could argume that the only type for which it makes sense to distinguish between the two is str itself -- indeed I rarely define different __repr__ and __str__ functions for new classes I create (but I do note that PEP 435 does define them differently for enum members). But for the str type, it's pretty important to have str(s) equal to s, and it's also pretty important to have a way to produce a string literal from a string value. And it would be annoying if we only had str()/__str__() as a general protocol and repr() was just a string method -- imagine the number of times people would be calling s.repr() in order to have unambiguous debug output only to get a traceback because s is None or some other non-str object... So it looks like we really need both str(x) and repr(x). But maybe we only need the __repr__ protocol? str(x) could just special-case its own type, and use repr(x) (or x.__repr__(), which is the same) in all other cases. The __repr__() method on the string type would do just what it does today. But there would not be a __str__() protocol at all. That would reduce some confusion and make the language spec a tiny bit smaller, and it might stop people from proposing/requesting that str() of a list should return something different than its repr(). It also would make it a little harder for some classes (like enums) to do something nicer when printed. But IIRC there are almost no builtin types that use this feature. Personally I think that "Color.red" still looks like debug output, intended for the developer of the program, not for its user. If I wanted to show a Color to an end user of my program, I'd be printing x.name anyway. And for debugging, "Color.red" suits me fine as well -- it even fulfills the (always informal!) rule of thumb that the repr() of an object should resemble an expression that has that object as a value better than the repr() currently defined in the PEP. After all, if I really wanted to know what was inside, I could always print x.value... Please don't see this as a proposal to change the language. Like Greg, I'm not advocating, just pondering. (With the exception that if I was allowed to use the time machine to go back a couple of weeks, I'd adjust PEP 435 to define str(x) as x.name, e.g. "red", and repr(x) as what is currently defined as str(x), e.g. "Color.red".) -- --Guido van Rossum (python.org/~guido)

7 7

PEP 442 delegate
by Antoine Pitrou May 23, 2013

May 23, 2013

Hello, I would like to nominate Benjamin as BDFL-Delegate for PEP 442. Please tell me if you would like to object :) Regards Antoine.

6 6

_PyString_InsertThousandsGrouping()
by Carlos Nepomuceno May 23, 2013

May 23, 2013

Hi guys! Can someone explain to me where in the CPython 2.7.5 source code is _PyString_InsertThousandsGrouping() implemented? I've found the following declaration in 'Objects/stringobject.c' but it just defines _Py_InsertThousandsGrouping() as _PyString_InsertThousandsGrouping(): "#define _Py_InsertThousandsGrouping _PyString_InsertThousandsGrouping" I'm looking for the opposite! I don't even know how that doesn't cause an error! What's the trick? Besides that I've found a lot of code inside some header files, such as 'Objects/stringlib/formatter.h'. Why did you chose that way? Thanks in advance. Carlos

2 3

Best practices for Enum
by Raymond Hettinger May 22, 2013

May 22, 2013

After the long design effort for the enum module, I'm sure there will be a forthcoming effort to apply them pervasively throughout the standard library. I would like to ask for a little restraint and for there to be individual cost/benefit evaluations for each case. On the plus-side, the new integer-enums have a better repr than plain integers. For internal constants such as those in idlelib and regex, the user won't see any benefit at all. But there will be a cost in terms of code churn, risk of introducing errors in stable code, modestly slowing-down the code, making it more difficult to apply bug fixes across multiple versions of Python, and increased code verbosity (i.e. changing "if direction=LEFT: ..." to "if direction is Direction.LEFT: ...") For external constants, some thought needs to be given to: * is the current API working just fine (i.e. decimal's ROUND_DOWN) * will enums break doctests or any existing user code * will it complicate users converting from Python 2 * do users now have to learn an additional concept * does it complicate the module in any way I'm hoping that enums get used only in cases where they clearly improve the public API (i.e. cases such as sockets that have a large number of integer constants) rather than having a frenzy of every constant, everywhere getting turned into an enum. I would like to see enums used as tool for managing complexity, rather than becoming a cause of added complexity by being used for every problem, the tall and small, even where it is not needed at all. my-two-cents-ly yours, Raymond

23 55

Is thread-safe smtpd desired/possible?
by Sorin Stelian May 21, 2013

May 21, 2013

Hi, I am posting this here since I could find no active maintainer of the smtpd module. In my work as a test engineer for Axis (www.axis.com) I encountered the need of having thread-safe SMTP servers. I know the use case of several SMTP servers running in concurrent threads might seem odd, but it can actually be quite useful for testing purposes. I have implemented (for my own use) a possible solution which basically means that every SMTP channel has its own socket map, instead of using asyncore's global socket map. It would not involve any change in asyncore. Looking at the disucssion from http://bugs.python.org/issue11959 it seems to me that such a solution would not be accepted. Do you think that modifying asyncore is more feasible? If not, is this something that might be looked at? I can provide code if needed, but I would first like to know your thoughts about this. Best regards, Sorin

2 1

Re: [Python-Dev] [Python-checkins] cpython (3.3): #17973: Add FAQ entry for ([], )[0] += [1] both extending and raising.
by Nick Coghlan May 21, 2013

May 21, 2013

On Tue, May 21, 2013 at 12:35 AM, r.david.murray <python-checkins(a)python.org> wrote: Yay for having this in the FAQ, but... > +If you wrote:: > + > + >>> a_tuple = (1, 2) > + >>> a_tuple[0] += 1 > + Traceback (most recent call last): > + ... > + TypeError: 'tuple' object does not support item assignment > + > +The reason for the exception should be immediately clear: ``1`` is added to the > +object ``a_tuple[0]`` points to (``1``), producing the result object, ``2``, > +but when we attempt to assign the result of the computation, ``2``, to element > +``0`` of the tuple, we get an error because we can't change what an element of > +a tuple points to. > + > +Under the covers, what this augmented assignment statement is doing is > +approximately this:: > + > + >>> result = a_tuple[0].__iadd__(1) > + >>> a_tuple[0] = result > + Traceback (most recent call last): > + ... > + TypeError: 'tuple' object does not support item assignment For the immutable case, this expansion is incorrect: >>> hasattr(0, "__iadd__") False With immutable objects, += almost always expands to: >>> result = a_tuple[0] + 1 >>> a_tuple[0] = result (For containers that support binary operators, the presence of the relevant __i*__ methods is actually a reasonable heuristic for distinguishing the mutable and immutable versions) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

PEP 435 - ref impl disc 2
by Glenn Linderman May 21, 2013

May 21, 2013

So I have a class based on Nick's Named Values, that has been extended to propagate names into expressions, so that if you have named values 'x' and 'y', when you x + y, the result is a named value whose name is '(x + y)'. Seems pretty awkward to integrate this with Enum. Maybe I'm missing something. Here's carved down code with just one operator defined for brevity. The third item from each print statement should be the same, as far as I understand... but isn't. class NamedInt( int ): _count = 0 def __new__( cls, *args, **kwds ): name, *args = args if len( args ) == 0: args = [ cls._count ] cls._count += 1 self = super().__new__( cls, *args, **kwds ) self._name = name return self def __init__( self, *args, **kwds ): name, *args = args super().__init__() @property def __name__( self ): return self._name def __repr__( self ): # repr() is updated to include the name and type info return "{}({!r}, {})".format(type(self).__name__, self.__name__, super().__repr__()) def __str__( self ): # str() is unchanged, even if it relies on the repr() fallback base = super() base_str = base.__str__ if base_str.__objclass__ is object: return base.__repr__() return base_str() # for simplicity, we only define one operator that propagates expressions def __add__(self, other): temp = int( self ) + int( other ) if isinstance( self, NamedInt ) and isinstance( other, NamedInt ): return NamedInt( '({0} + {1})'.format(self.__name__, other.__name__), temp ) else: return temp x = NamedInt('the-x', 1 ) y = NamedInt('the-y', 2 ) # demonstrate that NamedInt propagates the names into an expression syntax print( repr( x ), repr( y ), repr( x+y )) from ref435 import Enum # requires redundant names, but loses names in the expression class NEI( NamedInt, Enum ): x = NamedInt('the-x', 1 ) y = NamedInt('the-y', 2 ) print( repr( NEI( 1 )), repr( NEI( 2 )), repr( NEI(1) + NEI(2))) # looks redundant, and still loses the names in the expression class NEI2( NamedInt, Enum ): x = x y = y print( repr( NEI2( x )), repr( NEI2( x )), repr( NEI2(x) + NEI2(y)))

4 31

Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
by Olemis Lang May 20, 2013

May 20, 2013

---------- Forwarded message ---------- From: Olemis Lang <olemis(a)gmail.com> Date: Mon, 20 May 2013 11:33:42 -0500 Subject: Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum] To: Antoine Pitrou <solipsis(a)pitrou.net> On 5/20/13, Antoine Pitrou <solipsis(a)pitrou.net> wrote: > On Sat, 18 May 2013 23:41:59 -0700 > Raymond Hettinger <raymond.hettinger(a)gmail.com> wrote: >> >> We should continue to encourage users to make thorough unit tests >> and to leave doctests for documentation. That said, it should be >> recognized that some testing is better than no testing. And doctests >> may be attractive in that regard because it is almost effortless to >> cut-and-paste a snippet from the interactive prompt. That isn't a >> best practice, but it isn't a worst practice either. > > There are other reasons to hate doctest, such as the obnoxious > error reporting. Having to wade through ten pages of output to find > what went wrong is no fun. > +1 FWIW , while using dutest [1]_ each interactive example will be a test case and therefore the match for that particular assertion will be reported using the usual unittest output format .. [1] dutest (https://pypi.python.org/pypi/dutest) -- Regards, Olemis. Apache™ Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article:

1 0

Why is documentation not inline?
by Demian Brecht May 20, 2013

May 20, 2013

This is more out of curiosity than to spark change (although I wouldn't argue against it): Does anyone know why it was decided to document external to source files rather than inline? When rapidly digging through source, it would be much more helpful to see parameter docs than to either have to find source lines (that can easily be missed) to figure out the intention. Case in point, I've been digging through cookiejar.py and request.py to figure out their interactions. When reading through build_opener, it took me a few minutes to figure out that each element of *handlers can be either an instance /or/ a class definition (I was looking at how to define a custom cookiejar for an HTTPCookieProcessor). Yes, I'm (now) aware that there's some documentation at the top of request.py, but it would have been helpful to have it right in the definition of build_opener. It seems like external docs is standard throughout the stdlib. Is there an actual reason for this? Thanks, -- Demian Brecht http://demianbrecht.github.com

7 9

[RELEASED] Python 2.7.5
by Benjamin Peterson May 20, 2013

May 20, 2013

It is my greatest pleasure to announce the release of Python 2.7.5. 2.7.5 is the latest maintenance release in the Python 2.7 series. You may be surprised to hear from me so soon, as Python 2.7.4 was released slightly more than a month ago. As it turns out, 2.7.4 had several regressions and incompatibilities with 2.7.3. Among them were regressions in the zipfile, gzip, and logging modules. 2.7.5 fixes these. In addition, a data file for testing in the 2.7.4 tarballs and binaries aroused the suspicion of some virus checkers. The 2.7.5 release removes this file to resolve that issue. For details, see the Misc/NEWS file in the distribution or view it at http://hg.python.org/cpython/file/ab05e7dd2788/Misc/NEWS Downloads are at http://python.org/download/releases/2.7.5/ As always, please report bugs to http://bugs.python.org/ (Thank you to those who reported these bugs in 2.7.4.) This is a production release. Happy May, Benjamin Peterson 2.7 Release Manager (on behalf of all of Python 2.7's contributors)

8 15

Async subprocesses on Windows with tulip
by Richard Oudkerk May 19, 2013

May 19, 2013

Attached is a pretty trivial example of asynchronous interaction with a python subprocess using tulip on Windows. It does not use transports or protocols -- instead sock_recv() and sock_sendall() are used inside tasks. I am not sure what the plan is for dealing with subprocesses currently. Shall I just add this to the examples folder for now? -- Richard

2 2

HAVE_FSTAT?
by Antoine Pitrou May 19, 2013

May 19, 2013

Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks unconditional? It would avoid having to maintain unused fallback paths. Regards Antoine.

7 10

Re: [Python-Dev] cpython: Use PY_FORMAT_SIZE_T because Visual Studio does not understand %zd format.
by Serhiy Storchaka May 18, 2013

May 18, 2013

18.05.13 19:37, richard.oudkerk написав(ла): > http://hg.python.org/cpython/rev/0648e7fe7a72 > changeset: 83829:0648e7fe7a72 > user: Richard Oudkerk <shibturn(a)gmail.com> > date: Sat May 18 17:35:19 2013 +0100 > summary: > Use PY_FORMAT_SIZE_T because Visual Studio does not understand %zd format. See also DEBUG_PRINT_FORMAT_SPEC() in Python/formatter_unicode.c, _PyDebugAllocatorStats() in Objects/obmalloc.c, and kqueue_event_repr() in Modules/selectmodule.c.

1 1

Re: [Python-Dev] cpython: Undo the deprecation of _asdict().
by Serhiy Storchaka May 18, 2013

May 18, 2013

18.05.13 10:06, raymond.hettinger написав(ла): > http://hg.python.org/cpython/rev/1b760f926846 > changeset: 83823:1b760f926846 > user: Raymond Hettinger <python(a)rcn.com> > date: Sat May 18 00:05:20 2013 -0700 > summary: > Undo the deprecation of _asdict(). Why?

1 0

[RELEASED] Python 3.2.5 and Python 3.3.2
by Georg Brandl May 18, 2013

May 18, 2013

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax modules. Details can be found in the changelogs: http://hg.python.org/cpython/file/v3.2.5/Misc/NEWS and http://hg.python.org/cpython/file/v3.3.2/Misc/NEWS To download Python 3.2.5 or Python 3.3.2, visit: http://www.python.org/download/releases/3.2.5/ or http://www.python.org/download/releases/3.3.2/ respectively. As always, please report bugs to http://bugs.python.org/ (Thank you to those who reported these regressions.) Enjoy! - -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and all contributors) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlGUbJ4ACgkQN9GcIYhpnLDH8ACdEM4k7bobLJsFmCb49zuwQR3W EjgAoIWAOFNhJNdTAWEGSWqFWUP20wrb =YnPr -----END PGP SIGNATURE-----

7 6

Mysterious Python pyc file corruption problems
by Barry Warsaw May 18, 2013

May 18, 2013

I am looking into a particularly vexing Python problem on Ubuntu that manifests in several different ways. I think the problem is the same one described in http://bugs.python.org/issue13146 and I sent a message on the subject to the ubuntu-devel list: https://lists.ubuntu.com/archives/ubuntu-devel/2013-May/037129.html I don't know what's causing the problem and have no way to reproduce it, but all the clues point to corrupt pyc files in Pythons < 3.3. The common way this manifests is a traceback on an import statement. The actual error can be a "ValueError: bad marshal data (unknown type code)" such as in http://pad.lv/1010077 or an "EOFError: EOF read where not expected" as in http://pad.lv/1060842. We have many more instances of both of these. Since both error messages come from marshal.c when trying to read the pyc for a module being imported, I suspect that something is causing the pyc files to get partially overwritten or corrupted. The workaround is always to essentially blow away the .pyc file and re-create it. (Various different techniques can be used, but they all boil down to the same thing.) Another commonality is that this bug -- so far -- has not been observed in any Python 3.3 code, only 3.2 and earlier, including 2.7 and 2.6. This strengthens my hypothesis, since importlib in Python 3.3 included an atomic rename of the .pyc file whereas older Pythons only do an exclusive open on the pyc files, but do *not* do an atomic rename AFAICT. This leads me to hypothesize that the bug is due to an as yet unidentified race condition during installation of Python source code on Ubuntu, which is normally when we automatically byte compile the source to .pyc files. This can happen at package installation/upgrade time, or during a fresh install. In each of these cases there *should* be only one process attempting to write the .pyc, but my guess is that for some reason, multiple processes are trying to do this, triggering a truncation or other bogus content of .pyc files. Even in Python < 3.3, it should not be possible to corrupt a .pyc when only a single process is involved, due to the import lock and/or GIL. The exclusive open of the .pyc file is clearly not enough of a protection in a multiprocess situation, since the bug has already been identified in Python on buildbots during test_multiprocessing. See http://bugs.python.org/issue13146 I think the list of errors we've seen is too extensive to chalk up to a hardware bug, and I think the systems involved are modern enough to not be subject to file system data loss. There could be a missing fsync somewhere though that might be involved. I think it's doubtful that buggy remote file systems (e.g. NFSv2) are involved. I could be wrong about any of that. I have not succeeded in writing a standalone reproducer using Python 2.7. So, the mystery is: what process on Ubuntu is exploiting holes in the exclusive open and causing this problem? Issue 13146 is closed because the fix was applied to Python 3.3 (see above), but it was not backported to earlier versions. I think it would not be that difficult to backport it, and I would be willing to do so for Python 2.7 and 3.2. We might include 2.6 in that, but only in Ubuntu since I can't see how this bug could be exploited as a security vulnerability. Thoughts? -Barry

16 36

Summary of Python tracker Issues
by Python tracker May 17, 2013

May 17, 2013

ACTIVITY SUMMARY (2013-05-10 - 2013-05-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3966 ( +3) closed 25805 (+47) total 29771 (+50) Open issues with patches: 1776 Issues opened (32) ================== #17487: wave.Wave_read.getparams should be more user friendly http://bugs.python.org/issue17487 reopened by Claudiu.Popa #17807: Generator cleanup without tp_del http://bugs.python.org/issue17807 reopened by pitrou #17905: Add check for locale.h http://bugs.python.org/issue17905 reopened by pitrou #17951: TypeError during gdb backtracing http://bugs.python.org/issue17951 opened by Catalin.Patulea #17953: sys.modules cannot be reassigned http://bugs.python.org/issue17953 opened by Valentin.Lorentz #17955: Minor updates to Functional HOWTO http://bugs.python.org/issue17955 opened by akuchling #17956: add ScheduledExecutor http://bugs.python.org/issue17956 opened by neologix #17957: remove outdated (and unexcellent) paragraph in whatsnew http://bugs.python.org/issue17957 opened by tshepang #17960: Clarify the required behaviour of locals() http://bugs.python.org/issue17960 opened by ncoghlan #17961: Use enum names as values in enum.Enum convenience API http://bugs.python.org/issue17961 opened by ncoghlan #17963: Deprecate the frame hack for implicitly getting module details http://bugs.python.org/issue17963 opened by ncoghlan #17967: urllib2.open failed to access a url when a perent directory of http://bugs.python.org/issue17967 opened by foxkiller #17969: multiprocessing crash on exit http://bugs.python.org/issue17969 opened by kristjan.jonsson #17970: Mutlithread XML parsing cause segfault http://bugs.python.org/issue17970 opened by mrDoctorWho0.. #17972: inspect module docs omits many functions http://bugs.python.org/issue17972 opened by s7v7nislands(a)gmail.com #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 opened by pitrou #17975: libpython3.so conflicts between $VERSIONs http://bugs.python.org/issue17975 opened by prlw1 #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 opened by jasujm #17978: Python crashes if Py_Initialize/Py_Finalize are called multipl http://bugs.python.org/issue17978 opened by romuloceccon #17979: Cannot build 2.7 with --enable-unicode=no http://bugs.python.org/issue17979 opened by amaury.forgeotdarc #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 opened by fweimer #17984: io and _pyio modules require the _io module http://bugs.python.org/issue17984 opened by serhiy.storchaka #17985: multiprocessing Queue.qsize() and Queue.empty() with different http://bugs.python.org/issue17985 opened by aod #17986: Alternative async subprocesses (pep 3145) http://bugs.python.org/issue17986 opened by sbt #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 opened by fdrake #17988: ElementTree.Element != ElementTree._ElementInterface http://bugs.python.org/issue17988 opened by jwilk #17989: ElementTree.Element broken attribute setting http://bugs.python.org/issue17989 opened by jwilk #17991: ctypes.c_char gives a misleading error when passed a one-chara http://bugs.python.org/issue17991 opened by Steven.Barker #17994: Change necessary in platform.py to support IronPython http://bugs.python.org/issue17994 opened by icordasc #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 opened by giampaolo.rodola #17997: ssl.match_hostname(): sub string wildcard should not match IDN http://bugs.python.org/issue17997 opened by christian.heimes #17998: internal error in regular expression engine http://bugs.python.org/issue17998 opened by jdemeyer Most recent 15 issues with no replies (15) ========================================== #17998: internal error in regular expression engine http://bugs.python.org/issue17998 #17997: ssl.match_hostname(): sub string wildcard should not match IDN http://bugs.python.org/issue17997 #17996: socket module should expose AF_LINK http://bugs.python.org/issue17996 #17994: Change necessary in platform.py to support IronPython http://bugs.python.org/issue17994 #17991: ctypes.c_char gives a misleading error when passed a one-chara http://bugs.python.org/issue17991 #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 #17986: Alternative async subprocesses (pep 3145) http://bugs.python.org/issue17986 #17975: libpython3.so conflicts between $VERSIONs http://bugs.python.org/issue17975 #17942: IDLE Debugger: names, values misaligned http://bugs.python.org/issue17942 #17933: test_ftp failure / ftplib error formatting issue http://bugs.python.org/issue17933 #17924: Deprecate stat.S_IF* integer constants http://bugs.python.org/issue17924 #17923: test glob with trailing slash fail http://bugs.python.org/issue17923 #17916: Provide dis.Bytecode based equivalent of dis.distb http://bugs.python.org/issue17916 #17909: Autodetecting JSON encoding http://bugs.python.org/issue17909 #17902: Document that _elementtree C API cannot use custom TreeBuilder http://bugs.python.org/issue17902 Most recent 15 issues waiting for review (15) ============================================= #17988: ElementTree.Element != ElementTree._ElementInterface http://bugs.python.org/issue17988 #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 #17979: Cannot build 2.7 with --enable-unicode=no http://bugs.python.org/issue17979 #17978: Python crashes if Py_Initialize/Py_Finalize are called multipl http://bugs.python.org/issue17978 #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 #17956: add ScheduledExecutor http://bugs.python.org/issue17956 #17951: TypeError during gdb backtracing http://bugs.python.org/issue17951 #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 #17945: tkinter/Python 3.3.0: peer_create doesn't instantiate Text http://bugs.python.org/issue17945 #17944: Refactor test_zipfile http://bugs.python.org/issue17944 #17941: namedtuple should support fully qualified name for more portab http://bugs.python.org/issue17941 #17940: extra code in argparse.py http://bugs.python.org/issue17940 #17937: Collect garbage harder at shutdown http://bugs.python.org/issue17937 #17936: O(n**2) behaviour when adding/removing classes http://bugs.python.org/issue17936 Top 10 most discussed issues (10) ================================= #17914: add os.cpu_count() http://bugs.python.org/issue17914 36 msgs #17980: CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard http://bugs.python.org/issue17980 31 msgs #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 27 msgs #15392: Create a unittest framework for IDLE http://bugs.python.org/issue15392 16 msgs #17961: Use enum names as values in enum.Enum convenience API http://bugs.python.org/issue17961 15 msgs #17936: O(n**2) behaviour when adding/removing classes http://bugs.python.org/issue17936 14 msgs #17969: multiprocessing crash on exit http://bugs.python.org/issue17969 13 msgs #17976: file.write doesn't raise IOError when it should http://bugs.python.org/issue17976 10 msgs #8604: Adding an atomic FS write API http://bugs.python.org/issue8604 9 msgs #17974: Migrate unittest to argparse http://bugs.python.org/issue17974 9 msgs Issues closed (41) ================== #6208: path separator output ignores shell's path separator: / instea http://bugs.python.org/issue6208 closed by terry.reedy #14596: struct.unpack memory leak http://bugs.python.org/issue14596 closed by pitrou #17237: m68k aligns on 16bit boundaries. http://bugs.python.org/issue17237 closed by pitrou #17468: Generator memory leak http://bugs.python.org/issue17468 closed by pitrou #17547: "checking whether gcc supports ParseTuple __format__... " erro http://bugs.python.org/issue17547 closed by python-dev #17563: Excessive resizing of dicts when used as a cache http://bugs.python.org/issue17563 closed by rhettinger #17606: xml.sax.saxutils.XMLGenerator doesn't support byte strings http://bugs.python.org/issue17606 closed by serhiy.storchaka #17732: distutils.cfg Can Break venv http://bugs.python.org/issue17732 closed by georg.brandl #17742: Add _PyBytesWriter API http://bugs.python.org/issue17742 closed by haypo #17754: test_ctypes assumes LANG=C LC_ALL=C http://bugs.python.org/issue17754 closed by doko #17843: Lib/test/testbz2_bigmem.bz2 trigger virus warnings http://bugs.python.org/issue17843 closed by georg.brandl #17895: TemporaryFile name returns an integer in python3 http://bugs.python.org/issue17895 closed by terry.reedy #17906: JSON should accept lone surrogates http://bugs.python.org/issue17906 closed by serhiy.storchaka #17915: Encoding error with sax and codecs http://bugs.python.org/issue17915 closed by georg.brandl #17920: Documentation: "complete ordering" should be "total ordering" http://bugs.python.org/issue17920 closed by rhettinger #17927: Argument copied into cell still referenced by frame http://bugs.python.org/issue17927 closed by benjamin.peterson #17943: AttributeError: 'long' object has no attribute 'release' in Qu http://bugs.python.org/issue17943 closed by georg.brandl #17948: HTTPS and sending a big file size hangs. http://bugs.python.org/issue17948 closed by pitrou #17949: operator documentation mixup http://bugs.python.org/issue17949 closed by ezio.melotti #17950: Dynamic classes contain non-breakable reference cycles http://bugs.python.org/issue17950 closed by gvanrossum #17952: editors-and-tools section of devguide does not appear to be ac http://bugs.python.org/issue17952 closed by ned.deily #17954: Support creation of extensible enums through metaclass subclas http://bugs.python.org/issue17954 closed by ncoghlan #17958: int(math.log(2**i, 2)) http://bugs.python.org/issue17958 closed by mark.dickinson #17959: Alternate approach to aliasing for PEP 435 http://bugs.python.org/issue17959 closed by ncoghlan #17962: Broken OpenSSL version in Windows builds http://bugs.python.org/issue17962 closed by python-dev #17964: os.sysconf(): return type of the C function sysconf() is long, http://bugs.python.org/issue17964 closed by haypo #17965: argparse does not dest.replace('-', '_') for positionals http://bugs.python.org/issue17965 closed by r.david.murray #17966: Lack of consistency in PEP 8 -- Style Guide for Python Code http://bugs.python.org/issue17966 closed by gvanrossum #17968: memory leak in listxattr() http://bugs.python.org/issue17968 closed by pitrou #17971: Weird interaction between Komodo Python debugger C module & Py http://bugs.python.org/issue17971 closed by benjamin.peterson #17973: '+=' on a list inside tuple both succeeds and raises an except http://bugs.python.org/issue17973 closed by ronaldoussoren #17977: urllib.request.urlopen() cadefault argument is documented with http://bugs.python.org/issue17977 closed by barry #17981: SysLogHandler closes connection before using it http://bugs.python.org/issue17981 closed by vinay.sajip #17982: Syntax Error in IDLE3 not in IDLE http://bugs.python.org/issue17982 closed by terry.reedy #17983: global __class__ statement in class declaration http://bugs.python.org/issue17983 closed by python-dev #17990: 2.7 builds can fail due to unconditional inclusion of include http://bugs.python.org/issue17990 closed by benjamin.peterson #17992: test_asynchat hangs http://bugs.python.org/issue17992 closed by neologix #17993: Missed comma causes unintentional implicit string literal conc http://bugs.python.org/issue17993 closed by serhiy.storchaka #17995: report，中　高　层　管　理　技　能158766 http://bugs.python.org/issue17995 closed by fdrake #995907: memory leak with threads and enhancement of the timer class http://bugs.python.org/issue995907 closed by neologix #1662581: the re module can perform poorly: O(2**n) versus O(n**2) http://bugs.python.org/issue1662581 closed by gregory.p.smith

1 0

Why is nb_inplace_add copied to sq_inplace_concat?
by Matt Newell May 17, 2013

May 17, 2013

I have encountered what I believe to be a bug but I'm sure there is some reason things are done as they are and I am hoping someone can shed some light or confirm it is indeed a bug. As a bit of background I have a c++ class that I use sip to generate python bindings. The potential python bug manifests itself as: >>> rl = RecordList() >>> rl += [] >>> rl NotImplemented The bindings fill in nb_inplace_add which appears to be implemented properly, returning a new reference to Py_NotImplemented if the right hand argument is not as expected. Where things appear to go wrong is that PyNumber_InPlaceAdd, after getting a NotImplemented return value from nb_inplace_add, then attempts to call sq_inplace_concat. From reading the code it appears that sq_inplace_concat is not supposed to return NotImplemented, instead it should set an exception and return null if the right hand arg is not supported. In my case sq_inplace_concat ends up being the same function as nb_inplace_add, which results in the buggy behavior. When I figured this out I tried to find out why sq_inplace_concat was set to the same function as nb_inplace_add, and ended up having to set a watchpoint in gdb which finally gave me the answer that python itself is setting sq_inplace_concat during type creation in the various functions in typeobject.c. Stack trace is below. I don't really understand what the fixup_slot_dispatchers function is doing, but it does seem like there must be a bug either in what it's doing, or in PyNumber_InPlaceAdd's handling of a NotImplemented return value from sq_inplace_concat. Thanks, Matt Python 2.7.3 (default, Jan 2 2013, 13:56:14) [GCC 4.7.2] on linux2 Stack trace where a watch on sq->sq_inplace_concat reveals the change: Hardware watchpoint 5: *(binaryfunc *) 0xcf6f88 Old value = (binaryfunc) 0 New value = (binaryfunc) 0x7ffff4d41c78 <slot_RecordList___iadd__(PyObject*, PyObject*)> #0 update_one_slot.25588 (type=type@entry=0xcf6c70, p=0x86ba90) at ../Objects/typeobject.c:6203 #1 0x00000000004b96d0 in fixup_slot_dispatchers (type=0xcf6c70) at ../Objects/typeobject.c:6299 #2 type_new.part.40 (kwds=<optimized out>, args=0x0, metatype=<optimized out>) at ../Objects/typeobject.c:2464 #3 type_new.25999 (metatype=<optimized out>, args=0x0, kwds=<optimized out>) at ../Objects/typeobject.c:2048 #4 0x0000000000463c08 in type_call.25547 (type=0x7ffff65953a0, args=('RecordList', (<sip.wrappertype at remote 0x7ffff5201080>,), {'__module__': 'blur.Stone'}), kwds=0x0) at ../Objects/typeobject.c:721 #5 0x00000000004644eb in PyObject_Call (func=<type at remote 0x7ffff65953a0>, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2529

2 3

Re: [Python-Dev] [Python-checkins] cpython: fix compilation on Windows
by Terry Reedy May 17, 2013

May 17, 2013

On 5/16/2013 4:17 PM, victor.stinner wrote: > summary: fix compilation on Windows That fixed my problem with compiling 3.4, 32 bit, Win 7. Thanks. But I cannot compile 3.3 python_d since May 6. In fact, there are more errors now than 8 hours ago. 7 things failed to build instead of 5 (3 is normal for me, given the lack of some dependencies). I believe the following is new. Red error box with .../p33/PCBuild/make_versioninfo_d.exe is not a valid Win32 application. The VS gui output box has "Please verify that you have sufficient rights to run this command." Some more errors: 10>..\PC\pylauncher.rc(16): error RC2104: undefined keyword or key name: FIELD3 10> 9> symtable.c 9>..\Python\symtable.c(1245): error C2143: syntax error : missing ';' before 'type' 9>..\Python\symtable.c(1246): error C2065: 'cur' : undeclared identifier 9>..\Python\symtable.c(1248): error C2065: 'cur' : undeclared identifier 9>..\Python\symtable.c(1253): error C2065: 'cur' : undeclared identifier 23> Traceback (most recent call last): 23> File "build_ssl.py", line 253, in <module> 23> main() 23> File "build_ssl.py", line 187, in main 23> os.chdir(ssl_dir) 23> FileNotFoundError: [WinError 2] The system cannot find the file specified: '..\\..\\openssl-1.0.1e' Earlier, about 4 other files had several warnings. I do no see the warnings now because they compiled then and have not changed. Errors are more urgent, but should warnings be ignored? Terry

1 0