Mailman 3 June 2014 - Python-Dev

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com May 18, 2021

May 18, 2021

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: … [View More]$Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: [View Less]

8 14

Update to PEP 11 to clarify garnering platform support
by Brett Cannon Feb. 27, 2015

Feb. 27, 2015

Here is some proposed wording. Since it is more of a clarification of what it takes to garner support -- which is just a new section -- rather than a complete rewrite I'm including just the diff to make it easier to read the changes. *diff -r 49d18bb47ebc pep-0011.txt* *--- a/pep-0011.txt Wed May 14 11:18:22 2014 -0400* *+++ b/pep-0011.txt Fri May 16 13:48:30 2014 -0400* @@ -2,22 +2,21 @@ Title: Removing support for little used platforms Version: $Revision$ Last-Modified: $Date$ -… [View More]Author: martin(a)v.loewis.de (Martin von Löwis) +Author: Martin von Löwis <martin(a)v.loewis.de>, + Brett Cannon <brett(a)python.org> Status: Active Type: Process Content-Type: text/x-rst Created: 07-Jul-2002 Post-History: 18-Aug-2007 + 16-May-2014 Abstract -------- -This PEP documents operating systems (platforms) which are not -supported in Python anymore. For some of these systems, -supporting code might be still part of Python, but will be removed -in a future release - unless somebody steps forward as a volunteer -to maintain this code. +This PEP documents how an operating system (platform) garners +support in Python as well as documenting past support. Rationale @@ -37,16 +36,53 @@ change to the Python source code will work on all supported platforms. -To reduce this risk, this PEP proposes a procedure to remove code -for platforms with no Python users. +To reduce this risk, this PEP specifies what is required for a +platform to be considered supported by Python as well as providing a +procedure to remove code for platforms with little or no Python +users. +Supporting platforms +-------------------- + +Gaining official platform support requires two things. First, a core +developer needs to volunteer to maintain platform-specific code. This +core developer can either already be a member of the Python +development team or be given contributor rights on the basis of +maintaining platform support (it is at the discretion of the Python +development team to decide if a person is ready to have such rights +even if it is just for supporting a specific platform). + +Second, a stable buildbot must be provided [2]_. This guarantees that +platform support will not be accidentally broken by a Python core +developer who does not have personal access to the platform. For a +buildbot to be considered stable it requires that the machine be +reliably up and functioning (but it is up to the Python core +developers to decide whether to promote a buildbot to being +considered stable). + +This policy does not disqualify supporting other platforms +indirectly. Patches which are not platform-specific but still done to +add platform support will be considered for inclusion. For example, +if platform-independent changes were necessary in the configure +script which was motivated to support a specific platform that would +be accepted. Patches which add platform-specific code such as the +name of a specific platform to the configure script will generally +not be accepted without the platform having official support. + +CPU architecture and compiler support are viewed in a similar manner +as platforms. For example, to consider the ARM architecture supported +a buildbot running on ARM would be required along with support from +the Python development team. In general it is not required to have +a CPU architecture run under every possible platform in order to be +considered supported. Unsupporting platforms ---------------------- -If a certain platform that currently has special code in it is -deemed to be without Python users, a note must be posted in this -PEP that this platform is no longer actively supported. This +If a certain platform that currently has special code in Python is +deemed to be without Python users or lacks proper support from the +Python development team and/or a buildbot, a note must be posted in +this PEP that this platform is no longer actively supported. This note must include: - the name of the system @@ -69,8 +105,8 @@ forward and offer maintenance. -Resupporting platforms ----------------------- +Re-supporting platforms +----------------------- If a user of a platform wants to see this platform supported again, he may volunteer to maintain the platform support. Such an @@ -101,7 +137,7 @@ release is made. Developers of extension modules will generally need to use the same Visual Studio release; they are concerned both with the availability of the versions they need to use, and with keeping -the zoo of versions small. The Python source tree will keep +the zoo of versions small. The Python source tree will keep unmaintained build files for older Visual Studio releases, for which patches will be accepted. Such build files will be removed from the source tree 3 years after the extended support for the compiler has @@ -223,6 +259,7 @@ ---------- .. [1] http://support.microsoft.com/lifecycle/ +.. [2] http://buildbot.python.org/3.x.stable/ Copyright --------- [View Less]

3 4

More compact dictionaries with faster iteration
by Raymond Hettinger Jan. 3, 2015

Jan. 3, 2015

The current memory layout for dictionaries is unnecessarily inefficient. It has a sparse table of 24-byte entries containing the hash value, key pointer, and value pointer. Instead, the 24-byte entries should be stored in a dense table referenced by a sparse table of indices. For example, the dictionary: d = {'timmy': 'red', 'barry': 'green', 'guido': 'blue'} is currently stored as: entries = [['--', '--', '--'], [-8522787127447073495, 'barry', 'green'], … [View More] ['--', '--', '--'], ['--', '--', '--'], ['--', '--', '--'], [-9092791511155847987, 'timmy', 'red'], ['--', '--', '--'], [-6480567542315338377, 'guido', 'blue']] Instead, the data should be organized as follows: indices = [None, 1, None, None, None, 0, None, 2] entries = [[-9092791511155847987, 'timmy', 'red'], [-8522787127447073495, 'barry', 'green'], [-6480567542315338377, 'guido', 'blue']] Only the data layout needs to change. The hash table algorithms would stay the same. All of the current optimizations would be kept, including key-sharing dicts and custom lookup functions for string-only dicts. There is no change to the hash functions, the table search order, or collision statistics. The memory savings are significant (from 30% to 95% compression depending on the how full the table is). Small dicts (size 0, 1, or 2) get the most benefit. For a sparse table of size t with n entries, the sizes are: curr_size = 24 * t new_size = 24 * n + sizeof(index) * t In the above timmy/barry/guido example, the current size is 192 bytes (eight 24-byte entries) and the new size is 80 bytes (three 24-byte entries plus eight 1-byte indices). That gives 58% compression. Note, the sizeof(index) can be as small as a single byte for small dicts, two bytes for bigger dicts and up to sizeof(Py_ssize_t) for huge dict. In addition to space savings, the new memory layout makes iteration faster. Currently, keys(), values, and items() loop over the sparse table, skipping-over free slots in the hash table. Now, keys/values/items can loop directly over the dense table, using fewer memory accesses. Another benefit is that resizing is faster and touches fewer pieces of memory. Currently, every hash/key/value entry is moved or copied during a resize. In the new layout, only the indices are updated. For the most part, the hash/key/value entries never move (except for an occasional swap to fill a hole left by a deletion). With the reduced memory footprint, we can also expect better cache utilization. For those wanting to experiment with the design, there is a pure Python proof-of-concept here: http://code.activestate.com/recipes/578375 YMMV: Keep in mind that the above size statics assume a build with 64-bit Py_ssize_t and 64-bit pointers. The space savings percentages are a bit different on other builds. Also, note that in many applications, the size of the data dominates the size of the container (i.e. the weight of a bucket of water is mostly the water, not the bucket). Raymond [View Less]

27 70

Semi-official read-only Github mirror of the CPython Mercurial repository
by Eli Bendersky Oct. 15, 2014

Oct. 15, 2014

Hi all, https://github.com/python/cpython is now live as a semi-official, *read only* Github mirror of the CPython Mercurial repository. Let me know if you have any problems/concerns. I still haven't decided how often to update it (considering either just N times a day, or maybe use a Hg hook for batching). Suggestions are welcome. The methodology I used to create it is via hg-fast-export. I also tried to pack and gc the git repo as much as possible before the initial Github push - it went … [View More]

4 8

Reviving restricted mode?
by Guido van Rossum Aug. 14, 2014

Aug. 14, 2014

I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug: http://code.google.com/p/googleappengine/issues/detail?id=671 Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" … [View More]

19 35

Tracker Stats
by Ezio Melotti July 7, 2014

July 7, 2014

Hi, I added a new "stats" page to the bug tracker: http://bugs.python.org/issue?@template=stats The page can be reached from the sidebar of the bug tracker: Summaries -> Stats The data are updated once a week, together with the Summary of Python tracker issues. Best Regards, Ezio Melotti

6 11

PEP 471 -- os.scandir() function -- a better and faster directory iterator
by Ben Hoyt July 1, 2014

July 1, 2014

Hi Python dev folks, I've written a PEP proposing a specific os.scandir() API for a directory iterator that returns the stat-like info from the OS, the main advantage of which is to speed up os.walk() and similar operations between 4-20x, depending on your OS and file system. Full details, background info, and context links are in the PEP, which Victor Stinner has uploaded at the following URL, and I've also copied inline below. http://legacy.python.org/dev/peps/pep-0471/ Would love … [View More]feedback on the PEP, but also of course on the proposal itself. -Ben PEP: 471 Title: os.scandir() function -- a better and faster directory iterator Version: $Revision$ Last-Modified: $Date$ Author: Ben Hoyt <benhoyt(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-May-2014 Python-Version: 3.5 Abstract ======== This PEP proposes including a new directory iteration function, ``os.scandir()``, in the standard library. This new function adds useful functionality and increases the speed of ``os.walk()`` by 2-10 times (depending on the platform and file system) by significantly reducing the number of times ``stat()`` needs to be called. Rationale ========= Python's built-in ``os.walk()`` is significantly slower than it needs to be, because -- in addition to calling ``os.listdir()`` on each directory -- it executes the system call ``os.stat()`` or ``GetFileAttributes()`` on each file to determine whether the entry is a directory or not. But the underlying system calls -- ``FindFirstFile`` / ``FindNextFile`` on Windows and ``readdir`` on Linux and OS X -- already tell you whether the files returned are directories or not, so no further system calls are needed. In short, you can reduce the number of system calls from approximately 2N to N, where N is the total number of files and directories in the tree. (And because directory trees are usually much wider than they are deep, it's often much better than this.) In practice, removing all those extra system calls makes ``os.walk()`` about **8-9 times as fast on Windows**, and about **2-3 times as fast on Linux and Mac OS X**. So we're not talking about micro- optimizations. See more `benchmarks`_. .. _`benchmarks`: https://github.com/benhoyt/scandir#benchmarks Somewhat relatedly, many people (see Python `Issue 11406`_) are also keen on a version of ``os.listdir()`` that yields filenames as it iterates instead of returning them as one big list. This improves memory efficiency for iterating very large directories. So as well as providing a ``scandir()`` iterator function for calling directly, Python's existing ``os.walk()`` function could be sped up a huge amount. .. _`Issue 11406`: http://bugs.python.org/issue11406 Implementation ============== The implementation of this proposal was written by Ben Hoyt (initial version) and Tim Golden (who helped a lot with the C extension module). It lives on GitHub at `benhoyt/scandir`_. .. _`benhoyt/scandir`: https://github.com/benhoyt/scandir Note that this module has been used and tested (see "Use in the wild" section in this PEP), so it's more than a proof-of-concept. However, it is marked as beta software and is not extensively battle-tested. It will need some cleanup and more thorough testing before going into the standard library, as well as integration into `posixmodule.c`. Specifics of proposal ===================== Specifically, this PEP proposes adding a single function to the ``os`` module in the standard library, ``scandir``, that takes a single, optional string as its argument:: scandir(path='.') -> generator of DirEntry objects Like ``listdir``, ``scandir`` calls the operating system's directory iteration system calls to get the names of the files in the ``path`` directory, but it's different from ``listdir`` in two ways: * Instead of bare filename strings, it returns lightweight ``DirEntry`` objects that hold the filename string and provide simple methods that allow access to the stat-like data the operating system returned. * It returns a generator instead of a list, so that ``scandir`` acts as a true iterator instead of returning the full list immediately. ``scandir()`` yields a ``DirEntry`` object for each file and directory in ``path``. Just like ``listdir``, the ``'.'`` and ``'..'`` pseudo-directories are skipped, and the entries are yielded in system-dependent order. Each ``DirEntry`` object has the following attributes and methods: * ``name``: the entry's filename, relative to ``path`` (corresponds to the return values of ``os.listdir``) * ``is_dir()``: like ``os.path.isdir()``, but requires no system calls on most systems (Linux, Windows, OS X) * ``is_file()``: like ``os.path.isfile()``, but requires no system calls on most systems (Linux, Windows, OS X) * ``is_symlink()``: like ``os.path.islink()``, but requires no system calls on most systems (Linux, Windows, OS X) * ``lstat()``: like ``os.lstat()``, but requires no system calls on Windows The ``DirEntry`` attribute and method names were chosen to be the same as those in the new ``pathlib`` module for consistency. Notes on caching ---------------- The ``DirEntry`` objects are relatively dumb -- the ``name`` attribute is obviously always cached, and the ``is_X`` and ``lstat`` methods cache their values (immediately on Windows via ``FindNextFile``, and on first use on Linux / OS X via a ``stat`` call) and never refetch from the system. For this reason, ``DirEntry`` objects are intended to be used and thrown away after iteration, not stored in long-lived data structured and the methods called again and again. If a user wants to do that (for example, for watching a file's size change), they'll need to call the regular ``os.lstat()`` or ``os.path.getsize()`` functions which force a new system call each time. Examples ======== Here's a good usage pattern for ``scandir``. This is in fact almost exactly how the scandir module's faster ``os.walk()`` implementation uses it:: dirs = [] non_dirs = [] for entry in scandir(path): if entry.is_dir(): dirs.append(entry) else: non_dirs.append(entry) The above ``os.walk()``-like code will be significantly using scandir on both Windows and Linux or OS X. Or, for getting the total size of files in a directory tree -- showing use of the ``DirEntry.lstat()`` method:: def get_tree_size(path): """Return total size of files in path and subdirs.""" size = 0 for entry in scandir(path): if entry.is_dir(): sub_path = os.path.join(path, entry.name) size += get_tree_size(sub_path) else: size += entry.lstat().st_size return size Note that ``get_tree_size()`` will get a huge speed boost on Windows, because no extra stat call are needed, but on Linux and OS X the size information is not returned by the directory iteration functions, so this function won't gain anything there. Support ======= The scandir module on GitHub has been forked and used quite a bit (see "Use in the wild" in this PEP), but there's also been a fair bit of direct support for a scandir-like function from core developers and others on the python-dev and python-ideas mailing lists. A sampling: * **Nick Coghlan**, a core Python developer: "I've had the local Red Hat release engineering team express their displeasure at having to stat every file in a network mounted directory tree for info that is present in the dirent structure, so a definite +1 to os.scandir from me, so long as it makes that info available." [`source1 <http://bugs.python.org/issue11406>`_] * **Tim Golden**, a core Python developer, supports scandir enough to have spent time refactoring and significantly improving scandir's C extension module. [`source2 <https://github.com/tjguk/scandir>`_] * **Christian Heimes**, a core Python developer: "+1 for something like yielddir()" [`source3 <https://mail.python.org/pipermail/python-ideas/2012-November/017772.html>`_] and "Indeed! I'd like to see the feature in 3.4 so I can remove my own hack from our code base." [`source4 <http://bugs.python.org/issue11406>`_] * **Gregory P. Smith**, a core Python developer: "As 3.4beta1 happens tonight, this isn't going to make 3.4 so i'm bumping this to 3.5. I really like the proposed design outlined above." [`source5 <http://bugs.python.org/issue11406>`_] * **Guido van Rossum** on the possibility of adding scandir to Python 3.5 (as it was too late for 3.4): "The ship has likewise sailed for adding scandir() (whether to os or pathlib). By all means experiment and get it ready for consideration for 3.5, but I don't want to add it to 3.4." [`source6 <https://mail.python.org/pipermail/python-dev/2013-November/130583.html>`_] Support for this PEP itself (meta-support?) was given by Nick Coghlan on python-dev: "A PEP reviewing all this for 3.5 and proposing a specific os.scandir API would be a good thing." [`source7 <https://mail.python.org/pipermail/python-dev/2013-November/130588.html>`_] Use in the wild =============== To date, ``scandir`` is definitely useful, but has been clearly marked "beta", so it's uncertain how much use of it there is in the wild. Ben Hoyt has had several reports from people using it. For example: * Chris F: "I am processing some pretty large directories and was half expecting to have to modify getdents. So thanks for saving me the effort." [via personal email] * bschollnick: "I wanted to let you know about this, since I am using Scandir as a building block for this code. Here's a good example of scandir making a radical performance improvement over os.listdir." [`source8 <https://github.com/benhoyt/scandir/issues/19>`_] * Avram L: "I'm testing our scandir for a project I'm working on. Seems pretty solid, so first thing, just want to say nice work!" [via personal email] Others have `requested a PyPI package`_ for it, which has been created. See `PyPI package`_. .. _`requested a PyPI package`: https://github.com/benhoyt/scandir/issues/12 .. _`PyPI package`: https://pypi.python.org/pypi/scandir GitHub stats don't mean too much, but scandir does have several watchers, issues, forks, etc. Here's the run-down as of the stats as of June 5, 2014: * Watchers: 17 * Stars: 48 * Forks: 15 * Issues: 2 open, 19 closed **However, the much larger point is this:**, if this PEP is accepted, ``os.walk()`` can easily be reimplemented using ``scandir`` rather than ``listdir`` and ``stat``, increasing the speed of ``os.walk()`` very significantly. There are thousands of developers, scripts, and production code that would benefit from this large speedup of ``os.walk()``. For example, on GitHub, there are almost as many uses of ``os.walk`` (194,000) as there are of ``os.mkdir`` (230,000). Open issues and optional things =============================== There are a few open issues or optional additions: Should scandir be in its own module? ------------------------------------ Should the function be included in the standard library in a new module, ``scandir.scandir()``, or just as ``os.scandir()`` as discussed? The preference of this PEP's author (Ben Hoyt) would be ``os.scandir()``, as it's just a single function. Should there be a way to access the full path? ---------------------------------------------- Should ``DirEntry``'s have a way to get the full path without using ``os.path.join(path, entry.name)``? This is a pretty common pattern, and it may be useful to add pathlib-like ``str(entry)`` functionality. This functionality has also been requested in `issue 13`_ on GitHub. .. _`issue 13`: https://github.com/benhoyt/scandir/issues/13 Should it expose Windows wildcard functionality? ------------------------------------------------ Should ``scandir()`` have a way of exposing the wildcard functionality in the Windows ``FindFirstFile`` / ``FindNextFile`` functions? The scandir module on GitHub exposes this as a ``windows_wildcard`` keyword argument, allowing Windows power users the option to pass a custom wildcard to ``FindFirstFile``, which may avoid the need to use ``fnmatch`` or similar on the resulting names. It is named the unwieldly ``windows_wildcard`` to remind you you're writing power- user, Windows-only code if you use it. This boils down to whether ``scandir`` should be about exposing all of the system's directory iteration features, or simply providing a fast, simple, cross-platform directory iteration API. This PEP's author votes for not including ``windows_wildcard`` in the standard library version, because even though it could be useful in rare cases (say the Windows Dropbox client?), it'd be too easy to use it just because you're a Windows developer, and create code that is not cross-platform. Possible improvements ===================== There are many possible improvements one could make to scandir, but here is a short list of some this PEP's author has in mind: * scandir could potentially be further sped up by calling ``readdir`` / ``FindNextFile`` say 50 times per ``Py_BEGIN_ALLOW_THREADS`` block so that it stays in the C extension module for longer, and may be somewhat faster as a result. This approach hasn't been tested, but was suggested by on Issue 11406 by Antoine Pitrou. [`source9 <http://bugs.python.org/msg130125>`_] Previous discussion =================== * `Original thread Ben Hoyt started on python-ideas`_ about speeding up ``os.walk()`` * Python `Issue 11406`_, which includes the original proposal for a scandir-like function * `Further thread Ben Hoyt started on python-dev`_ that refined the ``scandir()`` API, including Nick Coghlan's suggestion of scandir yielding ``DirEntry``-like objects * `Final thread Ben Hoyt started on python-dev`_ to discuss the interaction between scandir and the new ``pathlib`` module * `Question on StackOverflow`_ about why ``os.walk()`` is slow and pointers on how to fix it (this inspired the author of this PEP early on) * `BetterWalk`_, this PEP's author's previous attempt at this, on which the scandir code is based .. _`Original thread Ben Hoyt started on python-ideas`: https://mail.python.org/pipermail/python-ideas/2012-November/017770.html .. _`Further thread Ben Hoyt started on python-dev`: https://mail.python.org/pipermail/python-dev/2013-May/126119.html .. _`Final thread Ben Hoyt started on python-dev`: https://mail.python.org/pipermail/python-dev/2013-November/130572.html .. _`Question on StackOverflow`: http://stackoverflow.com/questions/2485719/very-quickly-getting-total-size-… .. _`BetterWalk`: https://github.com/benhoyt/betterwalk Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

22 58

Fix Unicode-disabled build of Python 2.7
by Serhiy Storchaka June 29, 2014

June 29, 2014

I submitted a number of patches which fixes currently broken Unicode-disabled build of Python 2.7 (built with --disable-unicode configure option). I suppose this was broken in 2.7 when C implementation of the io module was introduced. http://bugs.python.org/issue21833 -- main patch which fixes the io module and adds helpers for testing. http://bugs.python.org/issue21834 -- a lot of minor fixes for tests. Following issues fix different modules and related tests: http://bugs.python.org/… [View More]

13 23

LZO bug
by MRAB June 27, 2014

June 27, 2014

Is this something that we need to worry about? Raising Lazarus - The 20 Year Old Bug that Went to Mars http://blog.securitymouse.com/2014/06/raising-lazarus-20-year-old-bug-that.…

2 1

buildbot.python.org down?
by Ned Deily June 27, 2014

June 27, 2014

The buildbot web site seems to have been down for some hours and still is as of 0915 UTC. I'm not sure who is watching over it but I'll ping the infrastructure team as well. -- Ned Deily, nad(a)acm.org

2 1

Summary of Python tracker Issues
by Python tracker June 27, 2014

June 27, 2014

ACTIVITY SUMMARY (2014-06-20 - 2014-06-27) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4643 (-12) closed 29004 (+72) total 33647 (+60) Open issues with patches: 2162 Issues opened (50) ================== #6916: Remove deprecated items from asynchat http://bugs.python.org/issue6916 reopened by ezio.melotti #10312: intcatcher() can deadlock http:… [View More]

1 0

Binary CPython distribution for Linux
by Gregory Szorc June 27, 2014

June 27, 2014

I'm an advocate of getting users and projects to move to modern Python versions. I believe dropping support for end-of-lifed Python versions is important for the health of the Python community. If you've done any amount of Python 3 porting work, you know things get much harder the more 2.x legacy versions you need to support. I led the successful charge to drop support for Python 2.6 and below from Firefox's build system. I failed to win the argument that Mercurial should drop 2.4 and 2.… [View More]5 [1]. A few years ago, I started a similar conversation with the LLVM project [2]. I wrote a blog post on the subject [3] that even got Slashdotted [4] (although I don't think that's the honor it was a decade ago). While much of the opposition to dropping Python <2.7 stems from the RHEL community (they still have 2.4 in extended support and 2.7 wasn't in a release until a few weeks ago), a common objection from the users is "I can't install a different Python" or "it's too difficult to install a different Python." The former is a legit complaint - if you are on shared hosting and don't have root, as easy as it is to add an alternate package repository that provides 2.7 (or newer), you don't have the permissions so you can't do it. This leaves users with attempting a userland install of Python. Personally, I think installing Python in userland is relatively simple. Tools like pyenv make this turnkey. Worst case you fall back to configure + make. But I'm an experienced developer and have a compiler toolchain and library dependencies on my machine. What about less experienced users or people that don't have the necessary build dependencies? And, even if they do manage to find or build a Python distribution, we all know that there's enough finicky behavior with things like site-packages default paths to cause many headaches, even for experienced Python hackers. I'd like to propose a solution to this problem: a pre-built distribution of CPython for Linux available via www.python.org in the list of downloads for a particular release [5]. This distribution could be downloaded and unarchived into the user's home directory and users could start running it immediately by setting an environment variable or two, creating a symlink, or even running a basic installer script. This would hopefully remove the hurdles of obtaining a (sane) Python distribution on Linux. This would allow projects to more easily drop end-of-life Python versions and would speed adoption of modern Python, including Python 3 (because porting is much easier if you only have to target 2.7). I understand there may be technical challenges with doing this for some distributions and with producing a universal binary distribution. I would settle for a binary distribution that was targeted towards RHEL users and variant distros, as that is the user population that I perceive to be the most conservative and responsible for holding modern Python adoption back. [1] http://permalink.gmane.org/gmane.comp.version-control.mercurial.devel/68902 [2] http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-December/056545.html [3] http://gregoryszorc.com/blog/2014/01/08/why-do-projects-support-old-python-… [4] http://developers.slashdot.org/story/14/01/09/1940232/why-do-projects-conti… [5] https://www.python.org/download/releases/2.7.7/ [View Less]

5 5

C version of functools.lru_cache
by Peter Brady June 26, 2014

June 26, 2014

Hello python devs, I was recently in need of some faster caching and thought this would be a good opportunity to familiarize myself with the Python/C api so I wrote a C extension for the lru_cache in functools. The source is at https://github.com/pbrady/fastcache.git and I've posted it as a package on PyPI (fastcache). There are some simple benchmarks on the github page showing about 9x speedup. I would like to submit this for incorporation into the standard library. Is there any interest … [View More]

2 2

Re: [Python-Dev] cpython (3.3): Closes #20872: dbm/gdbm/ndbm close methods are not documented
by Ned Deily June 26, 2014

June 26, 2014

In article <3gz1lK1lYkz7Lk0(a)mail.python.org>, jesus.cea <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/cf156cfb12e7 > changeset: 91398:cf156cfb12e7 > branch: 3.3 > parent: 91384:92d691c3ca00 > user: Jesus Cea <jcea(a)jcea.es> > date: Wed Jun 25 13:05:31 2014 +0200 > summary: > Closes #20872: dbm/gdbm/ndbm close methods are not documented The 3.3 branch is open only to security fixes. Please don'… [View More]

3 3

Python 2.7 patch levels turning two digit
by M.-A. Lemburg June 23, 2014

June 23, 2014

With PEP 466 and the constant flow of OpenSSL security fixes which are currently being handled via Python patch level releases, we will soon reach 2.7.10 and quickly go beyond that (also see http://bugs.python.org/issue21308). This opens up a potential backwards incompatibility with existing tools that assume the Python release version number to use the "x.y.z" single digit approach, e.g. code that uses sys.version[:5] for the Python version or relies on the lexicographic ordering of the … [View More]

13 39

subprocess shell=True on Windows doesn't escape ^ character
by anatoly techtonik June 22, 2014

June 22, 2014

I am banned from tracker, so I post the bug here: Normal Windows behavior: >hg status --rev ".^1" M mercurial\commands.py ? pysptest.py >hg status --rev .^1 abort: unknown revision '.1'! So, ^ is an escape character. See http://www.tomshardware.co.uk/forum/35565-45-when-special-command-line But subprocess doesn't escape it, making cross-platform command fail on Windows. ---[cut pysptest.py]-- import subprocess as sp # this fails with # abort: unknown revision '.1'! cmd = … [View More]

16 32

PyPy3 2.3.1 released
by Philip Jenvey June 20, 2014

June 20, 2014

===================== PyPy3 2.3.1 - Fulcrum ===================== We're pleased to announce the first stable release of PyPy3. PyPy3 targets Python 3 (3.2.5) compatibility. We would like to thank all of the people who donated_ to the `py3k proposal`_ for supporting the work that went into this. You can download the PyPy3 2.3.1 release here: http://pypy.org/download.html#pypy3-2-3-1 Highlights ========== * The first stable release of PyPy3: support for Python 3! * The stdlib has been … [View More]

4 3

Summary of Python tracker Issues
by Python tracker June 20, 2014

June 20, 2014

ACTIVITY SUMMARY (2014-06-13 - 2014-06-20) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4655 ( -7) closed 28932 (+73) total 33587 (+66) Open issues with patches: 2152 Issues opened (49) ================== #8110: subprocess.py doesn't correctly detect Windows machines http://bugs.python.org/issue8110 reopened by r.david.murray #21750: mock_open … [View More]

1 0

Criticism of execfile() removal in Python3
by Paul Sokolovsky June 19, 2014

June 19, 2014

Hello, I was pleasantly surprised with the response to recent post about MicroPython implementation details (https://mail.python.org/pipermail/python-dev/2014-June/134718.html). I hope that discussion means that posts about alternative implementations are not unwelcome here, so I would like to bring up another (of many) issues we faced while implementing MicroPython. execfile() builtin function was removed in 3.0. This brings few problems: 1. It hampers interactive mode - instead of short … [View More]

15 26

Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required
by Yates, Andy (CS Houston, TX) June 19, 2014

June 19, 2014

Python Dev, Andy here. I have a Windows product based on Python and I'm getting hammered to release a version that includes the fix in OpenSSL 1.0.1h. My product is built on a Windows system using Python installed from the standard Python installer at Python.org. I would be grateful if I could get some advice on my options. Will Python.org be releasing a Windows installer with the fix any time soon or will it be at the next scheduled release in November? If it is November, there's no way I … [View More]

8 11

Windows XP, Python 3.5 and PEP 11
by Victor Stinner June 17, 2014

June 17, 2014

Hi, I would like to know if Python 3.5 will still support Windows XP or not. Almost all flavors of Windows XP reached the end-of-life in April, 2014 except "Windows XP Embedded". There is even an hack to use Windows upgrades on the desktop flavor using the embedded flavor (by changing a key in the registry). Extracts of the Wikipedia page: "As of January 2014, at least 49% of all computers in China still ran XP. " "In January 2014, it was estimated that more than 95% of the 3 million … [View More]

4 4

Commit "avoid a deadlock with the interpreter head lock and the GIL during finalization"
by Victor Stinner June 17, 2014

June 17, 2014

Hi, I just saw a change in Python finalization related to threads. I'm not sure that it is correct to not call tstate_delete_common(). Is this change related to an issue? I don't see any specific test. --- changeset 91234:5ccb6901cf95 3.4 avoid a deadlock with the interpreter head lock and the GIL during finalization author Benjamin Peterson <benjamin(a)python.org> date Mon, 16 Jun 2014 23:07:49 -0700 (61 minutes ago) parents d1d1ed421717 children 2ed64ea19d81 fceb3a907260 files … [View More]

1 0

Why does _pyio.*.readinto have to work with 'b' arrays?
by Nikolaus Rath June 15, 2014

June 15, 2014

Hello, The _pyio.BufferedIOBase class contains the following hack to make sure that you can read-into array objects with format 'b': try: b[:n] = data except TypeError as err: import array if not isinstance(b, array.array): raise err b[:n] = array.array('b', data) I am now wondering if I should implement the same hack in BufferedReader (cf. issue 20578). Is there anything special about 'b' arrays that justifies … [View More]

4 7

Raspberry Pi Buildbot
by Tal Einat June 13, 2014

June 13, 2014

Is there one? If not, would you like me to set one up? I've got one at home with Raspbian installed not doing anything, it could easily run a buildslave. Poking around on buildbot.python.org/all/builders, I can only see one ARM buildbot[1], and it's just called "ARM v7". I also found this python-dev thread[2] along with a blog.python.org blog post[3] from 2012, which mentioned that Trent Nelson would be receiving a Raspberry Pi and setting up a buildslave on it. But I can't find mention of it … [View More]

2 1

Why does IOBase.__del__ call .close?
by Nikolaus Rath June 13, 2014

June 13, 2014

Hello, I recently noticed (after some rather protacted debugging) that the io.IOBase class comes with a destructor that calls self.close(): [0] nikratio@vostro:~/tmp$ cat test.py import io class Foo(io.IOBase): def close(self): print('close called') r = Foo() del r [0] nikratio@vostro:~/tmp$ python3 test.py close called To me, this came as quite a surprise, and the best "documentation" of this feature seems to be the following note (from the io library reference): "The abstract … [View More]

6 10

Backwards Incompatibility in logging module in 3.4?
by Don Spaulding June 13, 2014

June 13, 2014

Hi there, I just started testing a project of mine on Python 3.4.0b1. I ran into a change that broke compatibility with the logging module in 3.3. The basic test is: $ py34/bin/python -c 'import logging; print(logging.getLevelName("debug".upper()))' Level DEBUG $ py33/bin/python -c 'import logging; print(logging.getLevelName("debug".upper()))' 10 I quickly stumbled upon this webpage: http://aazza.github.io/2014/05/31/testing-on-multiple-versions-of-Python/ Which led me … [View More]

3 4

python-dev for MAC OS X
by Pedro Helou June 13, 2014

June 13, 2014

Hey, does anybody know how to install the python-dev headers and libraries for MAC OS X? -- Pedro Issa Helou Network Communication Engineering + 36 20 262 9274

3 2

Summary of Python tracker Issues
by Python tracker June 13, 2014

June 13, 2014

ACTIVITY SUMMARY (2014-06-06 - 2014-06-13) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4662 (+12) closed 28859 (+57) total 33521 (+69) Open issues with patches: 2150 Issues opened (52) ================== #15993: Windows: 3.3.0-rc2.msi: test_buffer fails http://bugs.python.org/issue15993 reopened by skrah #18910: IDle: test textView.py http://… [View More]

1 0

Re: [Python-Dev] [Idle-dev] KeyConfig, KeyBinding and other related issues.
by Saimadhav Heblikar June 13, 2014

June 13, 2014

Hi, I would like the keyseq validator to be reviewed. The diff file: https://gist.github.com/sahutd/0a471db8138383fd73b2#file-test-keyseq-diff A sample test runner file: https://gist.github.com/sahutd/0a471db8138383fd73b2#file-test-keyseq-runner… In its current form, it supports/has modifiers = ['Shift', 'Control', 'Alt', 'Meta'] alpha_uppercase = ['A'] alpha_lowercase = ['a'] direction = ['Up',] direction_key = ['Key-Up'] It supports validating combinations upto 4 in … [View More]length. Please test for the above set only. (It will extended easily to fully represent the respective complete sets. The reason it cant be done *now* is the due to how RE optionals are coded differently in my patch. See CLEANUP below). I will also add remaining keys like Backspace, Slash etc tomorrow. # Cleanup: If we decide to go ahead with RE validating keys as in the above patch, 0. I made the mistake of not coding RE optionals -> ((pat)|(pat)) same for all sets. The result is that, extending the current key set is not possible without making all RE optional patterns similar.(Read the starting lines of is_valid_keyseq method). 1. There is a lot of places where refactoring can be done and appropriate comment added. 2. I left the asserts as-is. They can be used in testing the validator method itself. 3. The above patch still needs support for Backspace, slash etc to be added. I decided to add, once I am sure we will use it. 4. I would like to know how it will affect Mac? What are system specific differences? Please run the test-runner script on it and do let me know. --- My friend told that this thing can be done by "defining a grammar and automata." I did read up about it, but found it hard to grasp everything. Can you say whether it would be easier to solve it that way than RE? Regards On 13 June 2014 17:15, Saimadhav Heblikar <saimadhavheblikar(a)gmail.com> wrote: > On 13 June 2014 16:58, Tal Einat <taleinat(a)gmail.com> wrote: >> On Fri, Jun 13, 2014 at 2:22 PM, Saimadhav Heblikar >> <saimadhavheblikar(a)gmail.com> wrote: >>> Just a heads up to both: I am writing a keyseq validator method. >>> It currently works for over 800 permutations of ['Shift', 'Control', >>> 'Alt', 'Meta', 'Key-a', 'Key-A', 'Up', 'Key-Up', 'a', 'A']. It works >>> for permutations of length 2 and 3. Beyond that its not worth it IMO. >>> I am currently trying to integrate it with test_configuration.py and >>> catching permutations i missed out. >>> >>> I post this, so that we dont duplicate work. I hope it to be ready by >>> the end of the day.(UTC +5.5) >> >> What is the method you are using? > > Regex. It is not something elegant. The permutations are coded in.(Not > all 800+ obviously, but around 15-20 general ones.). The only > advantage is it can be used without creating a new Tk instance. > > >> >> What do you mean by "permutations"? If you mean what I think, then I'm >> not sure I agree with >3 not being worth it. I've used keyboard >> bindings with more than 2 modifiers before, and we should certainly >> support this properly. >> > I am sorry. I meant to write >3 modifier permutations. > (i.eControl-Shift-Alt-Meta+Key-X is not covered. But > Control-Shift-Alt-Key-X is.) > > > > > -- > Regards > Saimadhav Heblikar -- Regards Saimadhav Heblikar [View Less]

1 1

Moving Python 3.5 on Windows to a new compiler
by Steve Dower June 13, 2014

June 13, 2014

Hi all I would like to propose moving Python 3.5 to use Visual C++ 14.0 as the main compiler. The first CTP of Visual Studio "14" was released earlier this week: http://blogs.msdn.com/b/vcblog/archive/2014/06/03/visual-studio-14-ctp.aspx The major feature of interest in this version of MSVC is a new policy to maintain binary compatibility for the CRT into the future. (There will be a blog about this soon, but I didn't want to hold up getting the discussion started here.) What this means for … [View More]

24 66

Issue #21205: add __qualname__ to generators
by Victor Stinner June 12, 2014

June 12, 2014

Hi, I'm working on asyncio and it's difficult to debug code because @asyncio.coroutine decorator removes the name of the function if the function is not a generator (if it doesn't use yield from). I propose to add new gi_name and gi_qualname fields to the C structure PyGenObject, add a new __qualname__ (= gi_qualname) attribute to the Python API of generator, and change how the default value of __name__ (= gi_name) of generators. Instead of getting the name from the code object, I propose to … [View More]

3 3

Returning Windows file attribute information via os.stat()
by Ben Hoyt June 11, 2014

June 11, 2014

Hi folks, As pointed out to me recently in an issue report [1] on my scandir module, Python's os.stat() simply discards most of the file attribute information fetched via the Win32 system calls. On Windows, os.stat() calls CreateFile to open the file and get the dwFileAttributes value, but it throws it all away except the FILE_ATTRIBUTE_DIRECTORY and FILE_ATTRIBUTE_READONLY bits. See CPython source at [2]. Given that os.stat() returns extended, platform-specific file attributes on Linux and … [View More]

7 14

Python 3.5 on VC14 - update
by Steve Dower June 10, 2014

June 10, 2014

For anyone who is interested in more details on the CRT changes, there's a blog post from my colleague who worked on most of them at http://blogs.msdn.com/b/vcblog/archive/2014/06/10/the-great-crt-refactoring… I wanted to call out one section and add some details: In order to unify these different CRTs [desktop, phone, etc], we have split the CRT into three pieces: 1. VCRuntime (vcruntime140.dll): This DLL contains all of the runtime functionality required for things like … [View More]process startup and exception handling, and functionality that is coupled to the compiler for one reason or another. We may need to make breaking changes to this library in the future. 2. AppCRT (appcrt140.dll): This DLL contains all of the functionality that is usable on all platforms. This includes the heap, the math library, the stdio and locale libraries, most of the string manipulation functions, the time library, and a handful of other functions. We will maintain backwards compatibility for this part of the CRT. 3. DesktopCRT (desktopcrt140.dll): This DLL contains all of the functionality that is usable only by desktop apps. Notably, this includes the functions for working with multibyte strings, the exec and spawn process management functions, and the direct-to-console I/O functions. We will maintain backwards compatibility for this part of the CRT. The builds of Python I've already made are indeed linked against these three DLLs, though it happens transparently. Most of the APIs are from the AppCRT, which is a good sign as it will simplify portability to other Windows-based platforms (though the direct references to the Win32 API will arise again to complicate this). Very few functions are imported from VCRuntime, which is the only part that *may* have breaking changes in the future (that's the current promise, and I'd expect it to be strengthened one way or the other by releas). Apart from the standard memcpy/strcpy type functions (which may be moved in later builds), these other imports are compiler helpers: * void terminate(void) (currently exported as a decorated C++ function, but that's going to be fixed) * __vcrt_TerminateProcess * __vcrt_UnhandledException * __vcrt_cleanup_type_info_names * _except_handler4_common * _local_unwind4 I've checked with our CRT dev and he says that these don't keep any state (and won't cause problems like we've seen in the past with FILE*), and are only there to deal with potential C++ exceptions - they are included at a point where it is impossible to tell whether C++ is involved, and so can't be removed. My builds pass almost all of regrtest.py and the only issues are with Tcl/tk and OpenSSL, which need to update their compiler version detection. I've built them with changes, though as usual Tcl/tk is a real pain. I ran a quick test with profile-guided optimization (PGO, pronounced "pogo"), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) Cheers, Steve [View Less]

5 6

Documentation Oversight
by Hasan Diwan June 10, 2014

June 10, 2014

>From the csv module pydoc: "The optional "dialect" parameter is discussed below" The discussion is actually above the method. Present in 2.7.6. -- H -- Sent from my mobile device Envoyé de mon portable

2 1

namedtuple implementation grumble
by Chris Withers June 10, 2014

June 10, 2014

Hi All, I've been trying to add support for explicit comparison of namedtuples into testfixtures and hit a problem which lead me to read the source and be sad. Rather than the mixin and class assembly in the function I expected to find, I'm greeted by an exec of a string. Curious as to what lead to that implementation approach? What does it buy that couldn't have been obtained by a mixin providing the functionality? In my case, that's somewhat irrelevant, I'm looking to store a comparer … [View More]

10 22

Help with the build system and my first patch
by Steven Stewart-Gallus June 9, 2014

June 9, 2014

Hello, I would like some help understanding the build system. I am currently working on an issue (http://bugs.python.org/issue21627) and plan to create some common functionality in Python/setcloexec.c and Include/setcloexec.h that is conditionally compiled in on POSIX systems and not on Windows systems. I need to extract this functionality out from _Py_set_inheritable because it needs to run in the dangerous context of right after a fork and I don't believe it can throw exceptions. How can I … [View More]

2 3

cpython and python debugger documentation
by Le Pa June 9, 2014

June 9, 2014

Hi, I am interested in learning how the cpython interpreter is designed and implemented, and also how the python debugger works internally. My ultimate purpose is to modify them for my distributed computing needs. Are there any documentations on these please? I have done some goggling but failed to find anything useful. Thanks you very much for your help! -Le

5 4

Re: [Python-Dev] [Python-checkins] cpython: Closes #21256: Printout of keyword args in deterministic order in mock calls.
by Berker Peksağ June 9, 2014

June 9, 2014

On Mon, Jun 9, 2014 at 11:16 AM, kushal.das <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/8e05e15901a8 > changeset: 91102:8e05e15901a8 > user: Kushal Das <kushaldas(a)gmail.com> > date: Mon Jun 09 13:45:56 2014 +0530 > summary: > Closes #21256: Printout of keyword args in deterministic order in mock calls. > > Printout of keyword args should be in deterministic order in > a mock function call. This will help to … [View More]write better doctests. > > files: > Lib/unittest/mock.py | 2 +- > Lib/unittest/test/testmock/testmock.py | 6 ++++++ > Misc/NEWS | 3 +++ > 3 files changed, 10 insertions(+), 1 deletions(-) > > > diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py > --- a/Lib/unittest/mock.py > +++ b/Lib/unittest/mock.py > @@ -1894,7 +1894,7 @@ > formatted_args = '' > args_string = ', '.join([repr(arg) for arg in args]) > kwargs_string = ', '.join([ > - '%s=%r' % (key, value) for key, value in kwargs.items() > + '%s=%r' % (key, value) for key, value in sorted(kwargs.items()) > ]) > if args_string: > formatted_args = args_string > diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py > --- a/Lib/unittest/test/testmock/testmock.py > +++ b/Lib/unittest/test/testmock/testmock.py > @@ -1206,6 +1206,12 @@ > with self.assertRaises(AssertionError): > m.hello.assert_not_called() > > + #Issue21256 printout of keyword args should be in deterministic order > + def test_sorted_call_signature(self): > + m = Mock() > + m.hello(name='hello', daddy='hero') > + text = "call(daddy='hero', name='hello')" > + self.assertEquals(repr(m.hello.call_args), text) Should this be assertEqual instead? --Berker > > def test_mock_add_spec(self): > class _One(object): > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -92,6 +92,9 @@ > Library > ------- > > +- Issue #21256: Printout of keyword args should be in deterministic order in > + a mock function call. This will help to write better doctests. > + > - Issue #21677: Fixed chaining nonnormalized exceptions in io close() methods. > > - Issue #11709: Fix the pydoc.help function to not fail when sys.stdin is not a > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins > [View Less]

1 0

[numpy wishlist] Interpreter support for temporary elision in third-party classes
by Nathaniel Smith June 7, 2014

June 7, 2014

Hi all, There's a very valuable optimization -- temporary elision -- which numpy can *almost* do. It gives something like a 10-30% speedup for lots of common real-world expressions. It would probably would be useful for non-numpy code too. (In fact it generalizes the str += str special case that's currently hardcoded in ceval.c.) But it can't be done safely without help from the interpreter, and possibly not even then. So I thought I'd raise it here and see if we can get any consensus on … [View More]whether and how CPython could support this. === The dream === Here's the idea. Take an innocuous expression like: result = (a + b + c) / c This gets evaluated as: tmp1 = a + b tmp2 = tmp1 + c result = tmp2 / c All these temporaries are very expensive. Suppose that a, b, c are arrays with N bytes each, and N is large. For simple arithmetic like this, then costs are dominated by memory access. Allocating an N byte array requires the kernel to clear the memory, which incurs N bytes of memory traffic. If all the operands are already allocated, then performing a three-operand operation like tmp1 = a + b involves 3N bytes of memory traffic (reading the two inputs plus writing the output). In total our example does 3 allocations and has 9 operands, so it does 12N bytes of memory access. If our arrays are small, then the kernel doesn't get involved and some of these accesses will hit the cache, but OTOH the overhead of things like malloc won't be amortized out; the best case starting from a cold cache is 3 mallocs and 6N bytes worth of cache misses (or maybe 5N if we get lucky and malloc'ing 'result' returns the same memory that tmp1 used, and it's still in cache). There's an obvious missed optimization in this code, though, which is that it keeps allocating new temporaries and throwing away old ones. It would be better to just allocate a temporary once and re-use it: tmp1 = a + b tmp1 += c tmp1 /= c result = tmp1 Now we have only 1 allocation and 7 operands, so we touch only 8N bytes of memory. For large arrays -- that don't fit into cache, and for which per-op overhead is amortized out -- this gives a theoretical 33% speedup, and we can realistically get pretty close to this. For smaller arrays, the re-use of tmp1 means that in the best case we have only 1 malloc and 4N bytes worth of cache misses, and we also have a smaller cache footprint, which means this best case will be achieved more often in practice. For small arrays it's harder to estimate the total speedup here, but 66% fewer mallocs and 33% fewer cache misses is certainly enough to make a practical difference. Such optimizations are important enough that numpy operations always give the option of explicitly specifying the output array (like in-place operators but more general and with clumsier syntax). Here's an example small-array benchmark that IIUC uses Jacobi iteration to solve Laplace's equation. It's been written in both natural and hand-optimized formats (compare "num_update" to "num_inplace"): https://yarikoptic.github.io/numpy-vbench/vb_vb_app.html#laplace-inplace num_inplace is totally unreadable, but because we've manually elided temporaries, it's 10-15% faster than num_update. With our prototype automatic temporary elision turned on, this difference disappears -- the natural code gets 10-15% faster, *and* we remove the temptation to write horrible things like num_inplace. What do I mean by "automatic temporary elision"? It's *almost* possible for numpy to automatically convert the first example into the second. The idea is: we want to replace tmp2 = tmp1 + c with tmp1 += c tmp2 = tmp1 And we can do this by defining def __add__(self, other): if is_about_to_be_thrown_away(self): return self.__iadd__(other) else: ... now tmp1.__add__(c) does an in-place add and returns tmp1, no allocation occurs, woohoo. The only little problem is implementing is_about_to_be_thrown_away(). === The sneaky-but-flawed approach === The following implementation may make you cringe, but it comes tantalizingly close to working: bool is_about_to_be_thrown_away(PyObject * obj) { return (Py_REFCNT(obj) == 1); } In fact, AFAICT it's 100% correct for libraries being called by regular python code (which is why I'm able to quote benchmarks at you :-)). The bytecode eval loop always holds a reference to all operands, and then immediately DECREFs them after the operation completes. If one of our arguments has no other references besides this one, then we can be sure that it is a dead obj walking, and steal its corpse. But this has a fatal flaw: people are unreasonable creatures, and sometimes they call Python libraries without going through ceval.c :-(. It's legal for random C code to hold an array object with a single reference count, and then call PyNumber_Add on it, and then expect the original array object to still be valid. But who writes code like that in practice? Well, Cython does. So, this is no-go. === A better (?) approach === This is a pretty arcane bit of functionality that we need, and it interacts with ceval.c, so I'm not at all confident about the best way to do it. (We even have an implementation using libunwind to walk the C stack and make sure that we're being called from ceval.c, which... works, actually, but is unsatisfactory in other ways.) I do have an idea that I *think* might work and be acceptable, but you tell me: Proposal: We add an API call PyEval_LastOpDefinitelyMatches(frame, optype, *args) which checks whether the last instruction executed in 'frame' was in fact an 'optype' instruction and did in fact have arguments 'args'. If it was, then it returns True. If it wasn't, or if we aren't sure, it returns False. The intention is that 'optype' is a semantic encoding of the instruction (like "+" or "function call") and thus can be preserved even if the bytecode details change. Then, in numpy's __add__, we do: 1) fetch the current stack frame from TLS 2) check PyEval_LastOpDefinitelyMatches(frame, "+", arg1, arg2) 3) check for arguments with refcnt == 1 4) check that all arguments are base-class numpy array objects (i.e., PyArray_CheckExact) The logic here is that step (2) tells us that someone did 'arg1 + arg2', so ceval.c is holding a temporary reference to the arguments, and step (3) tells us that at the time of the opcode evaluation there were no other references to these arguments, and step (4) tells us that 'arg1 + arg2' dispatched directly to ndarray.__add__ so there's no chance that anyone else has borrowed a reference in the mean time. AFAICT PyEval_LastOpDefinitelyMatches can *almost* be implemented now; the only problem is that stack_pointer is a local variable in PyEval_EvalFrameEx, and we would need it to be accessible via the frame object. The easy way would be to just move it in there. I don't know if this would have any weird effects on speed due to cache effects, but I guess we could arrange to put it into the same cache line as f_lasti, which is also updated on every opcode? OTOH someone has gone to some trouble to make sure that f_stacktop usually *doesn't* point to the top of the stack, and I guess there must have been some reason for this. Alternatively we could stash a pointer to stack_pointer in the frame object, and that would only need to be updated once per entry/exit to PyEval_EvalFrameEx. Obviously there are a lot of details to work out here, like what the calling convention for PyEval_LastOpDefinitelyMatches should really be, but: * Does this approach seem like it would successfully solve the problem? * Does this approach seem like it would be acceptable in CPython? * Is there a better idea I'm missing? -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org [View Less]

8 24

Re: [Python-Dev] [Python-ideas] Expose `itertools.count.start` and implement `itertools.count.__eq__` based on it, like `range`.
by Nick Coghlan June 7, 2014

June 7, 2014

On 7 June 2014 19:36, Ram Rachum <ram.rachum(a)gmail.com> wrote: > My need is to have an infinite immutable sequence. I did this for myself by > creating a simple `count`-like stateless class, but it would be nice if that > behavior was part of `range`. Handling esoteric use cases like it sounds yours was is *why* user defined classes exist. It does not follow that "I had to write a custom class to solve my problem" should lead to a standard library or builtin changing unless … [View More]

2 1

Division of tool labour in porting Python 2 code to 2/3
by Brett Cannon June 7, 2014

June 7, 2014

After Glyph and Alex's email about their asks for assisting in writing Python 2/3 code, it got me thinking about where in the toolchain various warnings and such should go in order to help direct energy to help develop whatever future toolchain to assist in porting. There seems to be three places where issues are/can be caught once a project has embarked down the road of 2/3 source compatibility: 1. -3 warnings 2. Some linter tool 3. Failing tests -3 warnings are things that we know … [View More]are flat-out wrong and do not cause massive compatibility issues in the stdlib. For instance, warning that buffer() is not in Python 3 is a py3k warning -- Glyph made a mistake when he asked for it as a new warning -- is a perfect example of something that isn't excessively noisy and won't cause issues when people run with it. But what about warning about classic classes? The stdlib is full of them and they were purposefully left alone for compatibility reasons. But there is a subtle semantic difference between classic and new-style classes, and so 2/3 code should consider switching (this is when people chime in saying "this is why we want a 2.8 release!", but that still isn't happening). If this were made a py3k warning in 2.7 then the stdlib itself would spew out warnings which we can't change due to compatibility, so that makes it not useful (http://bugs.python.org/issue21231). But as part of a lint tool specific to Python 2.7 that kind of warning would not be an issue and is easily managed and integrated into CI setups to make sure there are no regressions. Lastly, there are things like string/unicode comparisons. http://bugs.python.org/issue21401 has a patch from VIctor which warns when comparing strings and unicode in Python 2.7. Much like the classic classes example, the stdlib becomes rather noisy due to APIs that handle either/or, etc. But unlike the classic classes example, you just can't systematically verify that two variables are always going to be str vs. unicode in Python 2.7 if they aren't literals. If people want to implement type constraint graphs for 2.7 code to help find them then that's great, but I personally don't have that kind of time. In this instance it would seem like relying on a project's unit tests to find this sort of problem is the best option. With those three levels in mind, where do we draw the line between these levels? Take for instance the print statement. Right now there is no warning with -3. Do we add one and then update the 2.7 stdlib to prevent warnings being generated by the stdlib? Or do we add it to some linter tool to pick up when people accidentally leave one in their code? The reason I ask is since this is clear I'm willing to spearhead the tooling work we talked about at the language summit to make sure there's a clear path for people wanting to port which is as easy as (reasonably) possible, but I don't want to start on it until I have a clear indication of what people are going to be okay with. [View Less]

3 4

Internal representation of strings and Micropython
by Steven D'Aprano June 6, 2014

June 6, 2014

There is a discussion over at MicroPython about the internal representation of Unicode strings. Micropython is aimed at embedded devices, and so minimizing memory use is important, possibly even more important than performance. (I'm not speaking on their behalf, just commenting as an interested outsider.) At the moment, their Unicode support is patchy. They are talking about either: * Having a build-time option to restrict all strings to ASCII-only. (I think what they mean by that is … [View More]

29 127

asyncio/Tulip: use CPython as the new upstream
by Victor Stinner June 6, 2014

June 6, 2014

Hi, I added a new BaseEventLoop.is_closed() method to Tulip and Python 3.5 to fix an issue (see Tulip issue 169 for the detail). The problem is that I don't want to add this method to Python 3.4 because usually we don't add new methods in minor versions of Python (future version 3.4.2 in this case). Guido just wrote in the issue: "Actually for asyncio we have special dispensation to push new features to minor releases (until 3.5). Please push to 3.4 so the source code is the same everywhere (… [View More]

7 7

Summary of Python tracker Issues
by Python tracker June 6, 2014

June 6, 2014

ACTIVITY SUMMARY (2014-05-30 - 2014-06-06) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4650 (+15) closed 28802 (+52) total 33452 (+67) Open issues with patches: 2127 Issues opened (48) ================== #21614: Case sensitivity problem in multiprocessing. http://bugs.python.org/issue21614 opened by ColinPDavidson #21615: Curses bug report for … [View More]

1 0

Request: new "Asyncio" component on the bug tracker
by Victor Stinner June 5, 2014

June 5, 2014

Hi, Would it be possible to add a new "Asyncio" component on bugs.python.org? If this component is selected, the default nosy list for asyncio would be used (guido, yury and me, there is already such list in the nosy list completion). Full text search for "asyncio" returns too many results. Victor

2 1

Should standard library modules optimize for CPython?
by Steven D'Aprano June 4, 2014

June 4, 2014

I think I know the answer to this, but I'm going to ask it anyway... I know that there is a general policy of trying to write code in the standard library that does not disadvantage other implementations. How far does that go the other way? Should the standard library accept slower code because it will be much faster in other implementations? Briefly, I have a choice of algorithm for the median function in the statistics module. If I target CPython, I will use a naive but simple O(N log … [View More]

8 12

Some notes about MicroPython from an observer
by Daniel Holth June 4, 2014

June 4, 2014

- micropython is designed to run on a machine with 192 kilobytes of RAM and perhaps a megabyte of FLASH. The controller can execute read-only code directly from FLASH. There is no dynamic linker in this environment. (It also has a UNIX port). - However it does include a full Python parser and REPL, so the board can be programmed without a separate computer as opposed to, say, having to upload bytecode compiled on a regular computer. - It's definitely going to be a subset of Python. For example, … [View More]

1 0

%x formatting of floats - behaviour change since 3.4
by Chris Angelico June 3, 2014

June 3, 2014

I'm helping out with the micropython project and am finding that one of their tests fails on CPython 3.5 (fresh build from Mercurial this morning). It comes down to this: Python 3.4.1rc1 (default, May 5 2014, 14:28:34) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> "%x"%16.0 '10' Python 3.5.0a0 (default:88814d1f8c32, Jun 4 2014, 07:29:32) [GCC 4.7.2] on linux Type "help", "copyright", "credits" or "license" for more information. >… [View More]

4 5

use cases for "python-config" versus "pkg-config python"
by Michael Haubenwallner June 2, 2014

June 2, 2014

Hello! Stumbling over problems on AIX (Modules/python.exp not found) building libxml2 as python module let me wonder about the intended use-cases for 'python-config' and 'pkg-config python'. FWIW, I can see these distinct use cases here, and I'm kindly asking if I got them right: * Build an application containing a python interpreter (like python$EXE itself): + link against libpython.so + re-export symbols from libpython.so for python-modules (platform-specific) + This is similar to … [View More]

3 3

Updating turtle.py
by Terry Reedy June 2, 2014

June 2, 2014

I have two areas of questions about updating turtle.py. First the module itself, then a turtle tracker issue versus code cleanup policies. A. Unlike most stdlib modules, turtle is copyrighted and licensed by an individual. ''' # turtle.py: a Tkinter based turtle graphics module for Python # Version 1.1b - 4. 5. 2009 # Copyright (C) 2006 - 2010 Gregor Lingl # email: glingl(a)aon.at ''' I am not sure what the copyright covers other than the exact text contributed, with updates, by Gregor. It … [View More]certainly does not cover the API and whatever code he copied from the previous version (unless that was also by him, and I have no idea how much he copied when reimplementing). I don't think it should cover additions made by others either. Should there be another line to cover these? ''' # Permission is granted to anyone to use this software for any purpose, # including commercial applications, and to alter it and redistribute it # freely, subject to the following restrictions: # # 1. The origin of this software must not be misrepresented; you must not # claim that you wrote the original software. If you use this software # in a product, an acknowledgment in the product documentation would be # appreciated but is not required. # 2. Altered source versions must be plainly marked as such, and must not be # misrepresented as being the original software. # 3. This notice may not be removed or altered from any source distribution. ''' As to point 2, the source has been altered a bit (by others) but it is not marked as such. How should it be? ''' _ver = "turtle 1.1b- - for Python 3.1 - 4. 5. 2009" ''' Obsolete; delete or alter (how)? When this replaced the previous turtle.py, it was considered 'owned' by Gregor in that changes had to go through him. However, he became inactive soon after and maintenance ceased. There has been only one turtle-specific code change that I know of (by Ned Daily, a month ago, for OSX. So is turtle.py unpatchable by anyone or fair game for anyone? A particular example: Gregor added intermediate layers to isolate turtle from tkinter. (He had a plan to add other backends, but I believe he abandoned that.) If someone wanted to reduce the layering and make the code easier to understand and maintain, while speeding execution, would that be allowed now? B. Lets assuming that turtle.py is, at least to some degree, fair game for fixes and enhancements. PSF Python PyLadies (Jessica Keller, Lynn Root) are participating in the 2014 GNOME Outreach Program for Women (OPW) https://wiki.python.org/moin/OPW/2014 . One of the projects (bottem of that page) is Graphical Python, in particular Turtle. A few days ago, Jessica posted http://bugs.python.org/issue21573 Clean up turtle.py code formatting "Lib/turtle.py has some code formatting issues. Let's clean them up to make the module easier to read as interns start working on it this summer." She want to follow cleanup with code examination, fixes, and enhancements. Responding today, I cautioned that clean-up only patches, such as she apparently would like to start with, are not in favor. But I cannot say how the policy applies to her proposal. Does the 'promise' of later work on the code make preliminary clean-up ok? Since she only marked the issue for 3.5, I also cautioned that 3.5-only cleanups would make fixing bugs in other issues harder. Is the code clean-up policy the same for all branches? Test? you ask?. There are apparently no unittests for turtle. On the other hand, turtledemo tests the overall function of the module and of many (most) of the top-level turtle functions. -- Terry Jan Reedy [View Less]

5 10

[RELEASE] Python 2.7.7
by Benjamin Peterson June 1, 2014

June 1, 2014

I'm happy to announce the immediate availability of Python 2.7.7. Python 2.7.7 is a regularly scheduled bugfix release for the Python 2.7 series. This release includes months of accumulated bugfixes. All the changes in Python 2.7.7 are described in detail in the Misc/NEWS file of the source tarball. You can view it online at http://hg.python.org/cpython/raw-file/f89216059edf/Misc/NEWS The 2.7.7 release also contains fixes for two severe, if arcane, potential security vulnerabilities. The … [View More]

1 0